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EXECUTIVE SUMMARY 


INTERNATIONAL I&W 

Research on the Early Warning and Monitoring System 
(EWAMS). during FY 1980 is chronicled in this report. The 
proliferation of quantitative computer-based systems for 
facilitating I&W (Indications and Warning) analysis has pro¬ 
vided a significant impetus for EWAMS research. In addition, 
extensive evidence from behavioral decision theory indicates 
that humans are susceptible to various cognitive-perceptual 
biases and fallacies and that a computer-based tracking and 
warning system can be of value to intelligence analysts. 

Basic and applied political, social, and computer science 
R&D has clearly facilitated the process of designing, test¬ 
ing, and refining EWAMS. 

Initial EWAMS research revolved around the testing of 
quantitative political indicators, the development of general 
scans, and the selection of a computer base for the system. 
The various software and research tasks of the first phase 
of Early Warning and Monitoring Project efforts provided a 
firm foundation for FY 1980 activities. 

A key objective of the recent political research program 
involved the realm of crisis warning. The discriminant 
analysis methodology was tested extensively and subsequently 
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incorporated into the system. The accuracy and efficiency of 
the methodology were both impressive, leading to a decision 
to replace the simple extrapolative warning methodology with 
one based on discriminant function-generated probabilities. 

A second focus was the automated I&W module of EWAMS. 
Automated I&W consists of two components: alerts and directed 
scans. Alert scans automatically yield information concerning 
conflict "hotspots" (reports of the major centers of inter¬ 
national stress), monitoring alerts (lists of situations 
which warrant vigilant "watching" by the analyst), and warning 
alerts (lists of forecasted crises). Directed scans (e.g., 
historical search), in contrast, maximize flexibility from 
the perspective of the user of the system and emphasize the 
importance of the individual analyst's own knowledge, exper¬ 
tise, and wisdom. 

Comparative source performance constituted a third re¬ 
search task. Extensive comparisons of NYT ( New York Times ) 
and MAG ( Manchester Guardian ) were undertaken. Overall, the 
finding has been that MAG and NYT are two separate (albeit 
somewhat similar) "windows on the world." Although each has 
regional biases and strengths, the I&W analyst is advised in 
the typical instance to consult both EWAMS sources; this can 
be expected to yield a more reliable, valid portrait of reality 
and minimize both "misses" and "false alarms." 
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FY 80 software development and refinement activities were 


extensive in scope and magnitude. Numerous enhancements and 
cosmetic modifications have been made to EWAMS. The enhance¬ 
ments to the master version of the EWAMS, which is housed at 
DARPA's DDF (Demonstration and Development Facility), include 
keyword, SPI (Special Purpose Indicators), CAP (Country 
Activity Profile scores)—all new features of the analytic 
component of EWAMS—and automated I&W. The DDF/EWAMS, a 
minicomputer-based system in FORTRAN, was also converted to 
the tailored EUCOM/EWAMS, a microprocesser-based system in 
BASIC. 


INTRANATIONAL I&W 

Past basic and applied research on intranational stress 
and crisis has been exhaustively reviewed and assessed. An 
analytic system relevant to African intranational I&W should 
feature four primary modules: a data base; indicators; fore¬ 
casting methodologies; and a computer base. Progress on all 
four has been achieved, which provides the foundation for 
continued development and enhancement of the AWAMS. 

The prototype AWAMS is discussed in detail. The con¬ 
ceptual blueprint envisions a system which is user-oriented 
in nature, parallel to the existing EWAMS in design and 
operation, systematic and comprehensive in scope and depth, 
and short-term and real-time in focus and nature. 
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i 

The AWAMS data collection design has been completely 
developed (including subnational actors and targets and intra¬ 
national event categories). The current prototype AWAMS in¬ 
dicators are based on those available to the EWAMS users. 

Data have been amassed since the beginning of 1980 from three 
distinct sources: Manchester Guardian /London Times ; FBIS 
(Foreign Broadcast Information Service); and cable traffic. \ 

The software base of AWAMS has been designed and programmed. i 
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INTERNATIONAL I&W 





1.0 INTRODUCTION 


Recent research on the Early Warning and Monitoring 
System (EWAMS) is chronicled in this report. EWAMS, which 
is sponsored by the Cybernetics Technology Division of the 
Defense Advanced Research Projects Agency (DARPA/CTD), is 
being developed and tested at the International Public 
Policy Research Corporation (IPPRC). The testing is both 
retrospective and current (real-time) in nature. 

The focus of the research summarized in this volume— 
the dual and interrelated Indications and Warning (I&W) 
domains of international and intranational affairs monitor¬ 
ing and forecasting—represents the amalgamation of exten¬ 
sive work in computer science, political science, inter¬ 
national relations, and social scientific methodology. 
Considerable progress has occurred in the development and 
testing of the intern ational I&W tracking and warning 
system; work has recently been launched on a prototype 
version of an eventual intra national I&W system. Central to 
these endeavors is the conviction that recent and current 
R&D in the realms of I&W, quantitative political inquiry, 
and computer science provides a firm foundation for develop¬ 
ing a viable analytic system to aid the U.S. defense intel¬ 
ligence analysis community. 
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1.1 Background 


Two generalizations reinforce the desirability of the 
research from the vantage point of the I&W thrust. One is 
the proliferation of quantitative and computer-based systems 
for facilitating I&W analysis, a trend which is illuminated 
in Clarkson et al. (1980). The second is the extensive 
evidence from behavioral decision theory which demonstrates 
convincingly that, like all human information processers and 
decision makers, intelligence analysts are susceptible to a 
number of cognitive biases and fallacies of inference-making 
and reasoning in general. The experimental psychological 
and other pertinent literature on cognitive processes, ably 
summarized by Heuer (1980) and Stech (1979) and reported in 
a stream of journal articles and books, provides innumerable 
examples of judgmental heuristics and other tendencies which 
bias the tasks of information processing and information 
integration. 


1.1.1 Computerized systems . The first trend—the 
upsurge in computer-based, quantitative intelligence track¬ 
ing and/or warning systems—is indisputable. What is prob¬ 
lematic, however, is a twofold issue: the design, testing, 
and evaluation of such systems and the adoption and use of 
the systems by the target user community. The latter prob¬ 
lem is particularly vexing; machine technology has quite 
literally outpaced the human user of computers. 




Careful and successful design and testing programs are 
preludes to and determinants of the adoption of monitoring 
and warning analysis management systems. The validity and 
reliability of data bases, indicator systems, and method¬ 
ologies must be ascertained. In addition, the testing and 
evaluation processes must take into account the needs, 
interests, and other characteristics of users. 

Transfer of a monitoring and warning analysis manage¬ 
ment system obviously depends on the willingness of analysts 
to integrate the system into their repertoires of skills, 
aids, and approaches. As Clarkson et al. (1980) quite 
properly point out, one of the basic questions is that of 
how to aid the human mind in performing the warning analysis 
task; this issue must be confronted in the context of the 
realization that both humans and computers have limitations. 

Computers can store, process, retrieve, and display 
vast amounts of data quickly and efficiently; color graphics 
and other display technologies can be exploited to highlight 
key phenomena and otherwise assist the analyst. While the 
information retrieval and analytic capabilities of computers 
are impressive, machines neither can nor should be substi¬ 
tuted for expertise and judgment. 





1.1.2 Human constraints . At the same time, while 
the human analyst can identify the key variables and make 
ultimate decisions, he or she confronts inevitable cognitive 
constraints and other limitations. To cite one simple but 
very apt example, a computer can effortlessly search a data 
base of 100,000 items for the appearance of a certain pat¬ 
tern (e.g., the number of event/interaction sequences when 
accuse events preceded force events); the human analyst 
would have neither the time nor the ability to perform such 
a routine but often helpful pattern recognition operation. 
The analyst would simply tend to "recall" a few relevant 
examples, instances that in the typical case would be 
neither representative of nor randomly selected from the 
relevant universe. 

The development and refinement of the international I&W 
system which will be discussed in Section I of this study 
has reached the point that issues of performance have 
already arisen and have been dealt with in a preliminary 
fashion. Like almost all computer-based I&W systems, the 
testing and evaluation process has not been pursued in terms 
of a comprehensive, systematic research design. In the 
context of an explicit warning analysis model and a set of 
evaluative tools and criteria (see, e.g., Clarkson et al., 
1980), further efforts can be undertaken to test, evaluate, 
and refine the international I&W system; an analogous 







approach can also be applied to the more embryonic intrana¬ 
tional I&W system which is conceptually fleshed out in 
Section II. 

Central to the successful transfer of an I&W analysis- 
aiding system is an explicit emphasis on a user-oriented 
product. This concern has permeated the development of the 
international I&W system; a genuinely interactive system, 
computer graphics, ease of use, and simplicity of operation 
have all been emphasized. 

While the computer (software and hardware) base of the 
system has contributed to its user-oriented nature, it is 
equally important to demonstrate the validity and utility of 
the system to the user community. To an extent, of course, 
this has been the goal of recent and current phases of re¬ 
search; the testing of the data, indicator, and methodology 
bases of the system has been extensive and painstaking. How¬ 
ever, this aspect of the research effort must accelerate and 
must also undergo a qualitative transformation which in¬ 
corporates more directly and completely the question of 
assessing the utility of the system and its relevance to the 
user community; users will not adopt a system unless it is 
unequivocally demonstrated to them that it is useful and 
valid from their perspectives. 







Human information processing and cognitive limitations 


are pervasive in nature and extensive in scope; indirect 
evidence from social psychology/behavioral decision theory 
research and from other evaluation and decision contexts and 
direct evidence from post-mortems and other assessments of 
intelligence analysis support the validity of this general¬ 
ization. Intelligence analysts, in other words, are no more 
exempt from these biases than anyone else who engages in 
estimation and choice selection activities. 

A pervasive consistency tendency affects and often 
distorts perception and memory. The principle of mental 
economy favors both simplicity and stability; there is 
ordinarily a deeply rooted bias against the assimilation of 
information that would require attitude change. Information 
integration biases permeate the judgment process; for 
example, because of judgmental heuristics such as represent¬ 
ativeness and availability, people tend to make inferences 
that are too extreme given the data. Often, people even 
forget that they did not know something before it happened— 
the hindsight bias leads to a belief in foresight insights 
which they actually had only because of outcome knowledge 
(Stech, 1979). 


Computer-based monitoring and warning systems which 
combine retrospective with current data and indicators will 





not be a magic panacea for the array of cognitive-perceptual 
biases which intrude at the individual and organizational 
levels of analysis. But such systems may aid the human 
analyst in a variety of ways and thereby minimize such 
biases. 

For example, the availability heuristic, which entails 
the tendency to assume that those events are most probable 
for which the recall of relevant cases is easiest, is a 
common fallacy. There is also a frequent neglect of base 
rate data (the distribution of outcomes in similar situations) 
and an accompanying overemphasis on case data. Salience 
(the most recent or dramatic case/s) often determines the 
outcome of interpretation and judgmental processes. 

These tendencies pervade international affairs analysis. 
The overuse of the Munich analogy and the overapplication of 
the Cuban missile crisis as an exemplar are both illustra¬ 
tions of this syndrome. The League of Nations was a retro¬ 
spective effort to prevent World War I; the United Nations 
was similarly an after-the-fact response to the outbreak of 
World War II; current Soviet-Yugoslav relations are often 
compared to Soviet-Czech relations in 1968. These and an 
avalanche of other possible examples all indicate that case 
data are typically favored over base rate data. 









A computer-based warning system can mitigate against 
this bias by providing the analyst with an historical base 
against which to compare a current or emerging situation. 

Like less formal I&W analysis, quantitative monitoring and 
warning systems are structured around the principles of 
pattern analysis and the tracking of deviations. How 
deviant is a particular case? How deviant is the analogous 
case? What are the statistical patterns in terms of raw 
scores, deviation scores, frequencies, and probabilities? 
Quantitative, computer-based I&W systems cannot be expected 
to eradicate completely the problems and dangers of cogni¬ 
tive-perceptual biases; such systems can, however, minimize 
the potential scope of such constraints. 

1.1.3 R&D progress . Political and social science R&D 
has also experienced progress; without the foundation pro¬ 
vided by earlier basic research on political indicator 
systems, data bases, and conceptual frameworks, the past 
EWAMS research on computer-based monitoring and warning 
would have been impossible. 

Quantitative political science and international rela¬ 
tions have undergone a dual revolution in the last few decades. 
The quantitative/methodological thrust is one manifestation of 
this revolutionary process; statistics and sophisticated 
methodological strategies have permeated research in interna¬ 
tional politics and other fields of political science. 
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The second revolution has been cybernetic in nature; 
the advent of the large computer has facilitated the storage 
and analysis of massive data bases and the more recent trend 
toward the use of micro- and mini-computers encourages off¬ 
line/parallel and even on-line applications. With the 
upsurge in interactive systems, the use of graphics, and 
other illustrations of user-oriented software, even some 
"traditionalists" are beginning to recognize the virtues of 
computers. 

A third trend has also emerged. In addition to the 
proliferation of methodologies and the increased awareness— 
and appreciation—of an array of developments in computer 
hardware and software, a renewed concern with applied 
inquiry is clearly discernible. This is reflected in recent 
research programs in the Defense Advanced Research Projects 
Agency/Cybernetics Technology Division (DARPA/CTD). The 
very basic thrusts of the late 1960s and early 1970s have 
been superceded by an impressive number of less basic—and 
more applied--efforts. In the domain of crisis analysis and 
I&W, the extensive basic research is chronicled in Hopple 
and Rossa (1980) and the applied inquiry in Daly (1978). 

1.2 Problem Statement 


The Crisis Early Warning and Monitoring System (EWAMS) 
was initially designed to focus on three central problems: 
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• The development, evaluation, and improvement of 
procedures for monitoring and warning of inter¬ 
national security crises 

• The identification and observation of quantita¬ 
tive non-military indicators for crisis warning 

• The integration of quantitative indicators and 
methods into an interactive, user-oriented, 
computer-based crisis early warning and monitor¬ 
ing system 

In an attempt to resolve these and related problems, 
attention has been devoted to the development and testing of 
political indicators and the application of methodologies 
designed to enhance monitoring and lengthen lead time before 
crises. The results of progress in computer hardware/software 
and newly developed conceptual and analytical constructs and 
approaches have been and are being applied to an array of 
early warning problems. 

The problem of crisis early warning is a complex one and 
has been thoroughly described elsewhere. The Department of 
Defense (DoD) has recently allocated a great deal of atten¬ 
tion to the development of methods designed to enhance overall 
crisis warning and management capabilities. Currently, 
responsibility for the analysis, forecasting, management, and 
avoidance of future crises is spread across a number of 
governmental offices and agencies, including the Defense 
Intelligence Agency (DIA), the offices of Assistant Secretary 
of Defense for Intelligence (ASDI) and International Security 
Affairs (ASDISA), the Central Intelligence Agency (CIA), 
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the National Security Agency (NSA), and the Defense Science 
Advisory Board (DSB). Thus, the warning process encompasses 
various centers within the national security community. 

It is clear that in order to improve the warning time 
of intranational and international crises, technology for 
that purpose must be continually developed and tested. 
Furthermore, if the technology is intended for actual use, 
it must be tailored to the preferences and needs of those 
individuals who are participants in the warning process. 

The staff of the Early Warning and Monitoring Project has 
approached the warning problem by focusing on the devel¬ 
opment of technology to improve and facilitate the monitor¬ 
ing and warning processes. 

The complexity and sophistication of the warning pro¬ 
cess demonstrate that many international problems and crises 
encompass a much broader range of factors than military 
ones. Furthermore, research has suggested that, until these 
non-military factors (such as political and economic forces 
and trends) are identified and quantified for observation 
and testing, there is little likelihood that they will 
systematically and positively contribute to DoD's ability to 
monitor interactions and forecast crises to the extent that 
military indicators track affairs and facilitate the antici¬ 
pation of unusual situations and crises. Consequently, it is 
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quite possible that the lack of emphasis on quantitative 
work in the realm of political and other non-military 
indicators could seriously impair the ability of the United 
States to forecast and manage crises. This is because any 
final assessment of early warning, such as an overall judg¬ 
ment of the likelihood of hostile action, is critically 
dependent on political, social, economic, and military 
information. 

Since analysts are required to arrive at judgments 
regarding impending crisis events, it is critical that the 
analyst have access to appropriate technology to assist him 
in arriving at more accurate and valid judgments. Addi¬ 
tionally, the goals and priorities of the Department of 
Defense require a commitment to the improvement of the early 
warning process. This commitment necessitates continued 
development and testing of an integrated, user-oriented 
crisis warning system. Currently, there is a paucity of 
work on integrating quantitative indicators of intra- and 
international crises, developing warning systems based on 
political indicators, and developing user-oriented, com¬ 
puterized systems. 

Recognition of the importance of quantitative, non¬ 
military crisis indicators leads to an identification of 
several different types of related problems. Among the many 
factors which contribute to the overall warning problem are: 



• Supplementing warning systems with monitoring 
and retrieval systems 

• Coordinating and integrating the method¬ 
ologies and data of extant early warning 
projects and developing new ones 

• Tailoring early warning systems to user needs 

• Implementing an efficient and effective 
transfer of the system to the user community 


1.3 Toward a Solution 


Many important aspects of the initial problem cluster 
have been resolved. For example, the number and nature of 
quantitative indicators for crisis warning have been mea¬ 
sured as well as tested. Additionally, the indicators and 
forecasting methods now reside in an interactive, computer- 
based system. 

Potential solutions to the many remaining problems 
have been outlined. Strategies for completing the success¬ 
ful conceptual and methodological transition from a monitoring 
to a warning system have been identified. As new components 
are integrated into the system, they must be rigorously 
tested and evaluated. A significant amount of testing of 
quantitative political indicators has already been accom¬ 
plished; retrospective testing and testing with real-time 
data must be continued, as must interaction with potential 
users. To date, feedback from potential users who have seen 
the system has resulted in many valuable comments and 
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suggestions on substantive and cosmetic modifications which 
might make the system more useful and attractive to those in 
the I&W community. 


Under the terms of the DARPA/DIA Memorandum of Under¬ 
standing (MOU), the testing will be conducted at DIA/NMIC. 
The transfer to the European Command (EUCOM) site has also 
been initiated. Testing at other potential testbed sites 
will occur at the direction of the COTR. Such testing will 
facilitate and enhance the system's utility to those who 
will eventually work with it. 

While great progress continues to be made on the Crisis 
Early Warning and Monitoring System, the above indicates 
that much remains to be done. It is important that, as 
progress on a solution to the warning problem continues (a 
solution in the form of a multi-source, multi-indicator, 
multi-method, interactive computerized system), a broader 
problem is not ignored. The latter is the problem of inte¬ 
grating the results of current work being carried out by 
other DARPA/CTD contractors and transferring the final 
product to potential users at testbed sites. 

These multi-dimensional solutions to the problems 
associated with improved crisis early warning yield the 
following general research objectives: 
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computerized Crisis Early Warning and Monitoring System . 

This objective requires the continued development and inte¬ 
gration of international political and other (e.g., intra¬ 
national, expert-generated, etc.) indicators into the 
computerized early warning system. 

Objective 2 : Continue testing and evaluation of the 
Crisis Early Warning and Monitoring System . This objective 
requires continuation of the development of computer soft¬ 
ware and the application of hardware needed to implement the 
system. This objective also requires continuation of the 
highly successful interaction with potential users in order 
to tailor the system to their needs. 

Objective 4 : Integrate and transfer a user-oriented, 
multi-track, multi-source, multi-method Crisis Early Warning 
and Monitoring System . Once the development, testing, and 
evaluation processes are completed, it will be necessary to 
integrate the diverse results into the system. Transfer of 
the real-time early warning and monitoring system will also 
necessitate very close coordination with potential users at 
testbed locations. 

In connection with the objectives outlined above, the 
International Public Policy Research Corporation (IPPRC) 
plans to accomplish the following general tasks during FY 81. 
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Task 1 . Continue to collect and integrate data sets 


into the Early Warning and Monitoring System. Develop new 
indicators based on: region-specific international polit¬ 
ical event/interaction data; national capabilities data; and 
real-time, expert-generated data. The final integration of 
these data sets and indicators will make possible the reali¬ 
zation of a multi-source, multi-track, user-oriented early 
warning and monitoring system. 

Crucial to the successful completion of Task 1 is the 
continued real-time data collection, employing the New York 
Times and Manchester Guardian . The two sources are coded 
for international political event/interaction data and 
extend from January 1, 1966 to the present in the case of 
the former and from October 1, 1978 to the present in the 
case of the latter. In addition to maintaining and inte¬ 
grating data on a current, daily basis, quality control 
procedures will be maintained and implemented. 

Task 2 . In part based on the data sets integrated in 
Task 1, continue to develop the EWAMS by testing, evaluat¬ 
ing, and integrating political indicators. Develop new 
crisis indicators for daily and weekly data. Develop new 
crisis indicators based at least on rates of change, 
weights, and lead-lag considerations. Continue to develop 
the scanning capability of the system.. . 
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Task 3 . Continue retrospective testing of the EWAMS. 


In coordination with DIA/NMIC under the terms of the 
DARPA/DIA MOU, continue retrospective testing and evalua¬ 
tion of political indicators, scans, and probabilities. 

Task 4 . Continue to maintain, obtain, and integrate 
as much current and daily updated data as possible. Under 
the terms of the DARPA/DIA MOU, coordinate with DIA/NMIC 
to test and evaluate crisis indicators, scans, and proba¬ 
bilities in a real-time, off-line mode. Based in part on 
these tests and the input of DIA and EUCOM analysts, modify 
the EWAMS. At the direction of CTD, carry out periodic 
assessments of the quality of WEIS data collected on a daily 
basis. Where necessary, modify the format and content of 
analytic and descriptive WEIS data at regular intervals. 

Task 5 . Continue to identify, test, and evaluate new 
forecasting methods for crisis early warning. The fore¬ 
casting methods shall include: Markov renewal process 
models; stochastic, optimal control models; and real-time, 
expert-generated models. Evaluate and, where appropriate, 
integrate these forecasting methods into the EWAMS in order 
to enhance the system's multi-method forecasting and warning 
capability. 

Task 6 . Continue to develop and document interactive 
software for the EWAMS. Under the terms of the DARPA/DIA 
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MOU, seek input from DIA analysts and modify EWAMS software 
to meet their needs. To as great an extent as possible, 
continue to develop the hardware and software configuration 
to be compatible with those of other CTD contractors. 

Solicit DIA, EUCOM, and other user evaluations on the appro¬ 
priateness of all newly developed indicators and enhance¬ 
ments or modifications to current and potential users. In 
coordination with the staff at CTD's Demonstration and 
Development Facility (DDF), complete the software needed to 
successfully implement the automated I&W capability. Also 
in coordination with DDF, continue integrating color graph¬ 
ics and other display technology enhancements. 

Task 7 . Coordinate with DIA/NMIC and EUCOM to inte¬ 
grate and transfer each tested and evaluated component of 
the EWAMS to the DARPA/DIA Testbed and to EUCOM. In con¬ 
junction with DIA analysts at the Testbed, modify EWAMS as 
feasible to meet their needs. At the direction of CTD, 
assist in the transfer of the system to other potential 
users. Update and disseminate preliminary user's manuals as 
the system evolves. 

Task 8 . Continue to test and evaluate subjective 
methods for crisis warning. These should include, but not 
be limited to, the data and methods associated with real¬ 
time, off-line Markov renewal process and other appropriate 
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models developed by other CTD contractors. Develop and test 
procedures for integrating these and other expert assess¬ 
ments into the system. 
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2.0 THE EARLY WARNING AND MONITORING SYSTEM: 
AN OVERVIEW 


The philosophy which underlies the Early Warning and 
Monitoring System (EWAMS) is that human experts, the core of 
the I&W analytical process, can benefit from systems which 
supplement and enhance (but never replace) their intuition, 
judgment, and expertise. Ideally, such systems should reflect 
the fallible nature of any single source, indicator, and 
methodology. The EWAMS, for example, is multisource and 
features multiple indicators. There is also an array of fore¬ 
casting methodologies available to the analyst. 

Extensive research has been conducted on the Early 
** 

Warning and Monitoring System. This research has been con¬ 
cerned with the data, indicators, and probabilities which 
represent the key features of the overall system. The concept 
of an event undergirds the entire effort. 

Figure 2-1 portrays the elements of the concept of an 
event, a discrete and official verbal or physical action which 
is characterized in terms of who (actor) does what (event 

★ 

EWAMS is built around an objective/extrapolative warning 
methodology; other research efforts in the DARPA/CT program 
have focused on subjective and other methodologies. See 
especially Job and Duncan (1980). 

i** 

See Section 3.0 for a summary of EWAMS research. 
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type) to whom (target) when (date). The narrative or 
textual data are converted into quantitative data and the 
latter are in turn transformed into indicators (Figure 2-2). 

Event data may be extracted from any classified, private, 
or public source; the methodology of event data is uniform 
regardless of data source and consists of a set of systematic 
and replicable rules for converting verbal material into 
coded data. The current "master version" of the system relies 
on two sources; New York Times (NYT) and Manchester Guardian 
(MAG). The NYT data base extends from January 1, 1966 to the 
present and the MAG from October 1, 1978 to the present.* ** 

The event data and indicators are the foundation for 

EWAMS conflict probabilities. Currently offered only for 

monthly data (although work is proceeding on daily and weekly 

probabilities), the probabilities range from .01 to .99. 

They are generated from discriminant analysis, a social 

science methodology which classifies data into one of two 
* * 

categories. For example, the Internal Revenue Service uses 
discriminant analysis to classify tax returns into audit/do 
not audit categories. Analogously, a weather forecaster 
might use the methodology to classify data into rain/no rain. 

* 

The data are collected in virtual "real-time," which 
means that today's EWAMS includes NYT data up to yesterday. 

We have experimented with Reuters wire service data as well. 

** ’ 

Details are provided in Section 4.0 below. 
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The structure of the EWAMS is depicted in Figure 2-3. 


The analytic EWAMS is designed to provide a wealth of 
monitoring and warning indications to the analyst who is 
interested in scanning any international situation—especi¬ 
ally if the analyst wants to look in detail at a country 
pair. In contrast, the automated EWAMS scans a number of 
international situations and reports both global "hotspot 
lists" and monitoring and warning advice and alerts. The 
analytic and automated components are described in detail 
in Section 3.0. 


* 

There is a third module, which features Country Activity 
Profile (CAP) scores for any country and time period in the 
world. Each CAP score provides a reading for that country's 
activity as a proportion of world activity; the data are 
normalized or standardized so that the reading is relative to 
the country's past performance. For details, see Hopple and 
Snyder (1979). 
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3.0 SUMMARY OF RECENT RESEARCH 


This section summarizes the progress which has been made 
during this and earlier contract phases regarding the develop¬ 
ment, testing, and enhancement of an international I&W Crisis 
Early Warning and Monitoring System. The primary purpose of 
the research has been to develop and test an integrative and 
user-oriented early warning system. 

3.1 Initial Research 


An outgrowth of extensive basic and applied national 
security and international affairs research and analysis (much 
of it supported by DARPA/CTD), the Early Warning and Monitoring 
Project was launched in 1976. This section provides a summary 
of the research progress from 1976 through 1979; extended 
treatments of the various research and software findings and 
developments are available in Andriole (1976) , Bell et al. 
(1978), Daly and Davies (1978), and Wittmeyer (1976) and the 

structure and nature of the EWAMS as of June of 1979 is 

* 

detailed in a Sample Output package (IPPRC, 1979). 

The three core functions of the EWAMS—information 
retrieval, monitoring, and warning—have remained central 

A new Sample Output has since been published (IPPRC, 
1980c). 
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and are displayed in Figure 3-1. The underlying design of 
the system has been greatly influenced and clarified by a 
clearer conceptualization of the overall relationship between 
monitoring and warning processes (Martino, 1972). Two com¬ 
ponents of monitoring and warning have received a great deal 
of attention. These are indicator thresholds and the require¬ 
ment for current data. Thresholds are not a new system com¬ 
ponent but their importance has been greatly increased by 
two concerns: (1) requirements of the user community, and 
(2) the need to successfully make the transition from a 
monitoring to a warning system. For the EWAMS to be useful 
in an operational environment, its indicators must be based 
on current and regularly updated data. This requirement has 
influenced the amount of attention which we have given to the 
development of indicators included in the original design for 
a fully integrated EWAMS. 

Figure 3-1 

THREE SYSTEMS IN ONE 


INFORMATION 

RETRIEVAL 

+ 

MONITORING 

+ 

WARNING 





3.1.1 Quantitative indicators . Extensive testing has 
been performed on all components of the EWAMS. Such testing 
has been facilitated by the nature of the system's core data 
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base—the DARPA-supported World Event Interaction Survey 
(WEIS). WEIS data, which are collected on a daily basis and 
in a real-time mode at IPPRC, now consist of over 100,000 
non-routine international events for all countries of the 
world from 1966 to the present. The nature of the data base 
has permitted the testing of indicators, probabilities, and 
thresholds over a large number of historical crises. Daily 
updating of the data has been ongoing since the fall of 1979 
and has facilitated the most crucial aspect of testing—real¬ 
time monitoring and warning on a daily basis. 

All of the EWAMS indicators are derived from the WEIS 
events data. Quantitative indicators based on international 
events may be used to monitor international affairs through 
the "tracking" of indicator performance over time. Often, 
such monitoring results in the illumination of unusual changes 
or patterns which signify new problems and potential crises. 

Crisis warning is achieved through the integration of 
quantitative indicators and forecasting tools; aside from 
the analyst's forecasting expertise (intuitive skill), such 
other methodologies as probabilities or trend extrapolation 
techniques may employ the data provided by event-based indi¬ 
cators to warn of crises. 

The EWAMS features ten core political indicators . As 
displayed in Figure 3-2, indicators of volume and variety 
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Figure 3-2 

EWAMS CORE INDICATORS 



Raw 

Standardized 
(Z-Scored) 


Total Activity Level 

Total Activity Unusualness 


Cooperative Activity 

Cooperative Activity 

Volume 

Level 

Unusualness 


Conflict Activity 

Conflict Activity 


Level 

Unusualness 


Tension Level 

Tension Unusualness 

Variety 

\ 

Uncertainty (Hrel) 
Level 

Uncertainty Unusualness 


are available. Furthermore, each raw indicator (i.e., actual 
score) is accompanied by a standardized (statistically 
normalized) version which can be used to assess unusualness. 


The total activity level measures the sheer number of 
event/interactions exchanged. All political events are 
counted, regardless of WEIS category. This raw indicator of 
volume is useful because political activity levels tend to 
increase prior to (and during) a crisis between countries. 

Total activity is subdivided into the two general forms 
of cooperative activity and conflictual activity; in the WEIS 
coding scheme, event categories 1 through 10 are designated 
"cooperative" and the remaining categories 11 through 22 are 
defined as "conflictual." The cooperative activity level is 
the number of exchanged events coded as cooperative; the 
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conflictual activity level is the number of events coded as 
conflictual. 


Just as total activity increases prior to and during a 
crisis, conflict and cooperation also increase. However, 
the two sub-type of activity offer diagnostic capabilities 
beyond that of the total activity level indicator; by decom¬ 
posing overall activity into more discrete and interpretable 
components, the nature of the interactions is further illu¬ 
minated. 

The variety indicator of tension level utilizes the con¬ 
flict/cooperation division of events in order to provide a 
more qualitative measurement: the relative "conflictualness" 
of interactions. Tension level is defined by the algorithm: 

Tension _ Conflict activity level _ Conflict activity level 
Level Total activity level Total activity levels 

Thus, the tension level is the percentage of events which 
is conflictual; the percentage is adjusted to consider the 
inflationary effects of low total activity levels. Tension 
levels may vary from 0 (no conflict activity) to 100 (all 
activity is conflictual). Tension levels tend to increase 
to 70 or more prior to and during crises and conflict activity 
begins to predominate over cooperative activity—even as both 
increase. 
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Uncertainty, or Hrel, levels tap another form of variety 


which tends to increase in times of crisis. This measure 
uses the 22-category breakdown of WEIS events in order to as¬ 
sess the degree to which activity is dispersed across the many 
forms. To the extent that activity is equally distributed 
among all possible categories, uncertainty in events is high. 
The exact formula is drawn from communications theory: 

22 

Hrel - 2 P " 
n=l 

where P R is the proportion of all events which are in category 
n. When uncertainty is highest, each category has an equal 
number of events (1/22 of the total) and Hrel equals one; 
uncertainty is lowest when all events are of one type, result¬ 
ing in an Hrel of zero. Uncertainty in events tends to in¬ 
crease in periods of crisis, reaching levels of 0.7 or more. 

Although each of these indicators is useful in a system 
designed to monitor and warn of international crises, research 
has demonstrated that warning capability is enhanced when in¬ 
dicators are standardized or z-scored. Standardization in¬ 
volves the transformation of a raw indicator into an unusual¬ 
ness score, which takes into consideration the historical per¬ 
formance of an indicator for a particular dyadic (two-country) 
relationship, i.e., the indicator reading is adjusted to 


-log P n 
-log (1/22) 
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reflect the degree of unusualness when compared to prior 
readings. 

The formula for the conflict activity unusualness score 
will serve as an example of the transformation; the formula 
does not vary across raw indicators: 


Average of Previous 

Conflict Activity - Conflict Activity 

Conflict Activity _ _ Level __ Level _ 

Unusualness Standard Deviation (Dispersion of Pre- 

vious Conflict Activity Levels) 

The words Conflict Activity may be changed to any other indi¬ 
cator ( Total Activity / Cooperative Activity , Tension , or 
Uncertainty ) in order to provide its unusualness score formula. 

The unusualness score indicates the degree to which the 
current indicator reading differs from past readings. An 
unusualness score will equal zero if the current reading is 
"average" compared to the past. To the extent that the 
reading is unusually high, the unusualness score will be 
high; "unusually high" refers to a situation in which the 
reading is higher than average and more so than expected on 
the basis of past variations (standard deviation). Conversely, 
an unusualness score may be a large negative number when a 
reading is unusually low compared to the past. Unusualness 
indicators can therefore range from very large negative values 
to very large positive values (especially in precrisis and 
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crisis periods); the scores tend to approach zero in normal 
situations. 


In addition to the delineation of the core dyadic 
(country pairs) indicators, early EWAMS research, summarized 
in Andriole (1976) and Daly and Davies (1978), concentrated on 
other potential approaches and strategies. For example, sin¬ 
gle country activity was explored in a preliminary fashion; 
work focused on the ROZ (row percentage and column z-scores) 
indicator. ROZ takes into account a country's daily, weekly, 
or other (monthly, etc.) portion of total world action and the 
degree to which that portion is statistically exceptional 
compared to a ten-year average. 

ROZ proved to be an effective warning sign of impending 
danger, a monitor of on-going trouble, and an indicator of 
post-crisis shock (McClelland, 1976). As such, ROZ contrib¬ 
utes to all three functions of the system—information re¬ 
trieval, monitoring, and warning. ROZ was used to help deter¬ 
mine which countries should be added to a sample for develop¬ 
ing and testing indicators of intranational conflict and of 
intra- and international conflict linkages (Daly and Hopple, 
1977). 


3.1.2 General scans . General scans, i.e., tracking 
groups of countries that reflect user interests, are designed 
to enhance analyst efficiency. More specifically, a scan is 




an aggregation of countries by one or more criteria, e.g., a 
scan defined by geographic location aggregates countries into 
groups such as a Kiddle Eastern region (Wilkenfeld et al., 
1978) . 

A general scanning capability improves the monitoring 
and warning performance of the system in many ways. A general 
scan allows an analyst to look at several countries taken 
together rather than looking singly at many country pairs. 

If the indicators for the group suggest unusually high 
activity, tension, or uncertainty, the analyst can drop to 
the country-by-country level or even track the recent activity 
of a single country to determine the source of the disturbance 
on the regional level. More generally, scans can be viewed as 
having four primary purposes: 

• To increase analyst efficiency 

• To lengthen the warning time before crisis 

• To monitor and provide a characterization of 
the relationship between country pairs and 
the world as a whole 

• To facilitate and enhance crisis management 

The master version of the EWAMS is now front-ended with 
JCS regions. This particular scan allows the analyst to 
track a JCS region, to look at any combination of countries 
within that region, any combination of countries outside the 
region, any country within any region, and to create special 
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purpose user-tailored regions. Extensive research revealed 
the utility of employing JCS and other regional scans. 


3.1.3 Computer base . The master version of the Crisis 
Early Warning and Monitoring System's computer base is now 
located at DARPA/CTD's Demonstration and Development 
Facility (DDF). The status of the early warning system hard¬ 
ware and software can be summarized as: 

1. Hardware 

a. PDP 11/70 minicomputer 

b. Tektronix 4027, 4051, and 4052 graphic 
terminals 

c. Tektronix 4631 hard copy units 

d. Tektronix 4907 floppy disc 

2. Software 

a. UNIX operating system 

b. CULC's Fortran IV Plus 

c. Tektronix Plot 10 

d. Binary and random access data files 

Early project software also ran on a standalone 
Tektronix 4051. Standalone software programs were developed 
for testing new political indicators such as ROZ as well as 
for evaluating the warning capability of extant political 
indicators like the tension measure. 
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3.2 FY 1980 Research 


The software and political research and development 
processes of the early phase of the project provided a firm 
foundation for FY 80 research and analysis activities. In 
addition to the continued testing and development of 
quantitative indicators, this phase has also been charac¬ 
terized by an intensive (albeit preliminary) real-time test 
and evaluation program. Furthermore, there have been numerous 
extensions and enhancements of earlier thrusts as well as 
several distinctive lines of inquiry. 

Among the latter are various research efforts designed 
to improve and increase the rigor, validity, and reliability 
of the crisis warning component of the EWAMS. Secondly, the 
automated I&W module was converted from a blueprint into a 
detailed design and has been substantially programmed and 
implemented. The other major component of EWAMS—what is 
known as analytic I&W—was significantly expanded and enhanced. 
Dual source assessments and software development and implemen¬ 
tation represent tv/o additional areas where progress has been 
sustained. 

3.2.1 Crisis warning . During the early phases of 
development, EWAMS relied on an associative-extrapolative 
crisis forecasting methodology. Figure 3-3 depicts the 
range of forecasting methodologies potentially available 
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to the I&W analyst. The continuum extends from intuitive 
to very " objective 11 methodologies. The initial EWAMS fore¬ 
casting procedure was associative or extrapolative in 
nature; it contained a base of patterned cases (a conflict 
or crisis file) and viewed future crises an analogies to past 
ones. Activity, cooperation, and conflict probabilities in 
EWAMS reflected a prediction of the unknown (the short-term 
future) from the known (prior patterns). 

Figure 3-3 

THE FORECASTING METHODOLOGY TREE* 



Source: Armstrong (1978) . 












During FY 80, an alternative procedure—discriminant 
analysis—was incorporated into the EWAMS and replaced the 


simple extrapolative crisis warning methodology. This major 
change had been preceded by an extensive program of retro¬ 
spective testing and evaluation. 

In order to test the accuracy of the discriminant 
methodology for crisis warning, an historical data base 
covering 1966-1978 was constructed; thirty crises, including 
forty-eight nation pairs, were included and provided with a 
start-date. Using these crisis (start-) months and the 
eleven prior months in each case, the warning accuracy of the 
initial EWAMS indicators/probabilities was assessed: 

• Overall, a credible 90 percent of the forecasts 
provided by the initial EWAMS indicators/ 
probabilities are correct 

• The "average" crisis forecast is correct in 54 
percent of the cases and forecasts 33 percent of 
the actual crises 

• Current warning capability is weak when a one- 
month lead/lag before the crisis is utilized 

The discriminant analysis results were comparable and 
employed all indicators, as well as a dual approach which per¬ 
mitted "conservative" forecasting. The results included the 
following: 

• Overall success remains credible: 86 percent to 
92 percent of the forecasts are correct (depending 
upon the desired degree of conservatism) 
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• The "average" crisis forecast is correct in 
30 percent of the cases (50 percent when 
conservative), and 55 percent of the crises 
are forecast (20 percent when conservative) 

• Warning capability is only slightly improved 
when a one-month lead/lag is introduced, with 
performance below desired levels 

• The "average" crisis forecast is correct in 
30 percent of the cases (50 percent when 
conservative), and 55 percent of the crises 
are forecast (20 percent when conservative) 

• Warning capability is only slightly improved 
when a one-month lead/lag is introduced, with 
performance below desired levels 


Thus, the use of discriminant analysis for the purpose 
of estimating crisis probabilities resulted in forecast 
accuracies no less impressive than those previously gener¬ 
ated. The efficiency of the methodology recommended its use 
within the EWAMS. Furthermore, extensions made possible via 
the new method portend increased accuracy; for example, vari¬ 
ous time increments and z-score tails may be employed and 
multiple indicators may enter into probability calculations. 

3.2.2 Automated I&W . Hopple et al. (1979) discuss the 
detailed blueprint for the automated I&W module of EWAMS; 
this section offers an overview and provides an account of 
the extensive software and research progress. 

If the user selects automated I&W, he will initially 
choose one of two general options: alerts versus directed 
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scans. A developed alert option is already available. 
Directed scans will feature three distinct subroutines: 

• Networks 

• Forecasts 

• Historical searches 

The third is already operational and the other two com¬ 
ponents of directed scans will be developed in the future. 

Alert scans are defined by the system rather than by 
the user; all reporting criteria are pre-determined for the 
purpose of providing the I&W analyst the most meaningful 
information in a standardized mode. (The analyst can specify 
his own criteria in the directed scanning option.) The 
system-defined criteria reflect those prior (and ongoing) 
research findings which pinpoint the most productive thres¬ 
holds and methods for isolating meaningful alerts for 
monitoring international affairs and anticipating security 
crises. 

The existing prototype is patterned on the real-time 
assessment criteria. That is, tracking or monitoring indica¬ 
tors and thresholds are: 

• Tension, Hrel, and Conflict probability 
thresholds =50, .5, .5 
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Total and cooperative activity thresholds = 
.5 


The warning indicators are: 

• Tension, 70 

• Hrel, .7 

• Conflict probability, .7 

The prototype is designed to limit the required user- 
inputs (see Figure 3-4). The long-term plan provides that 
eventually no inputs will be solicited from users; the system 
will perform all conceivable scans and the time parameter will 
automatically be "real-time" or current. Such fully automated 
alert scans will complement the analytic EWAMS as well as the 
directed scans module. 

The current output includes: 

• "Hotspots" (reports of the most serious con¬ 
flict levels) 

• Monitoring alerts (lists of situations which, 
as a result of unusual events, warrant more 
detailed scrutiny and more vigilant "watching" 
by the analyst) 

• Warning alerts (lists of forecasted crises) 

A future enhancement. As Figure 3-4 indicates, will be 
threat networks. Threat networks will permit the analyst 
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Figure 3-4 


AUTOMATED ALERT STANS 



to map out those situations which are more explosive in nature 
This subsystem of the alert scans option will facilitate the 
tracking of crisis and conflict spillover processes and will 
enable the analyst to discern potential crisis linkages. 


* 

For example, the concept of second-order crises (crises 
not initially involving the concerned country) poses an array 
of challenges to international affairs researchers, policy 
analysts, and decisionmakers; the future international envi¬ 
ronment can be expected to provide U.S. I&W analysts and 
policy makers with a disturbingly large number of second-order 
crises which involve U.S. interests directly or indirectly. 
Recent examples include the Horn of Africa and Southeast Asia. 
Many of the Middle East crises exemplify potential crisis 
linkages; the 1973 Yom Kippur War, which "diffused" from the 
Arab-Israeli regional arena to the Soviet-U.S. "alert crisis," 
exemplifies this syndrome. 
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Conflict, cooperation, and "attention” networks are among the 
ones which could be offered to the user. 

The alert scans prototype is heavily indebted to prior 
research accomplishments. The criteria for discriminating 
between potential crises and situations which are unlikely to 
erupt into crises can be expected to show appreciable improve¬ 
ments in performance as research progresses. The emphasis on 
attempting to achieve "hits" and correct rejections while 
avoiding "misses" (incorrect crisis predictions) and false 
alarms will continue to provide a set of exacting standards 
for evaluating indicators and thresholds. 

Directed scans, the alternative option, maximize flexi¬ 
bility from the perspective of the user and emphasize the 
importance of the individual analyst's own knowledge, exper¬ 
tise, and wisdom. A sophisticated analyst can define his/ 
her criteria for scans and alerts. 

As Figure 3-5 suggests, these two automated I&W options, 
which will be developed simultaneously, seek to maximize 
orthogonal or contradictory goals. The linkage between 
alert scans and the historical search component of directed 
scans can be found in the parameter specification list: 
historical searches will accommodate any parameter specifica¬ 
tions and will yield sets of user-tailored output, whereas 







Figure 
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alert scans utilize all of the possible parameter settings to 
maximize the range and enhance the utility of alerts. 


Each envisioned sub-module of the directed scans option 
overcomes one or more of the limitations associated with 
other parts of the EWAMS blueprint. The result is a sub¬ 
system which requires both "intelligent" data-processing 
methods (as in alert scans) and "intelligent" or sophisticated 
user-supplied inputs. 

The historical search feature compensates for the limita¬ 
tions which characterize alert scans (Figure 3-6). The 
sophisticated user may be interested in obtaining "alerts" 
which are derived not from previous research results, but from 
the analyst's own knowledge base and interests. The time frame 
may be adjusted in order to observe the effects of historical 
situations or trends; scans can be limited to one or a few 
country pairs of interest; and indicators and thresholds may 
be designated by the analyst. The profiles which result from 
such user-defined specifications of criteria for alerts will 
thus automatically be tailored to the needs and interests of 
the particular analyst. 

3.2.3 Analytic I&W enhancements . The Early Warning and 
Monitoring System is designed to aid and facilitate the 
analysis of interactions between or among countries for the 
purpose of assessing the likelihood of crisis or serious 
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Figure 3-6 

DIRECTED SCANS: HISTORICAL SEARCH 



conflict. The system queries the user for information which 
defines the input parameters : 

• Actors (e.g., USA and IRN) 

• Flow (e.g., USA toward IRN) 

• Time increment (e.g., monthly) 

• Time frame (e.g., 79/01/01 through 79/12/31) 

In this example, the user asks to look at political 
actions (recorded in the New York Times ) directed from the 
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United States toward Iran during 1979, with the information 
aggregated at the monthly level (i.e., twelve reporting 
periods). This example is depicted in Figure 3-7, which is 
a reproduction of actual system output. 

The parameter specifications permit flexible "scanning" 
of international activity profiles. The user may examine any 
dyadic case (i.e., any country-to-country, region-to-region, 
or country-to-region set of actions or interactions). Each 
case may be analyzed for various time increments in order to 
observe daily, weekly, monthly, or yearly trends. 

The major function of the analytic EWAMS is to provide 
meaningful indications, a variety of monitoring devices, and 
probabilistic warnings regarding the present trends of the 
case selected by the user. 

The analytic EWAMS has continued to emphasize the 
originally posited four central components of the initial 
design: 


• General scans 

• Quantitative political indicators for monitor¬ 
ing and warning 

• A unified multi-method forecasting capability 

• A computer base 
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Figure 3-7 


SAMPLE INPUT FOR THE ANALYTIC EWAMS 

****** EARLY WARNING AND MONITORING SYSTEM ACTIVATED ****** 


Ar« your actors: 

1. Countries 

2. JCS regions 

3. Both 1 


Please select two countries<usa>usr>: usa>irn 


Specify activity flow: 

0. one way (usa >>> irn) 

1. one way <usa <<< irn) 

2. two way (usa <-> irr»> 0 


Select tiMe increment: 

1. daily 

2. weekly 

3. Monthly 

4. quarterly 

5. yearly 


3 


Set time parameters<750101-771231>: 


790101-791231 


However, the needs of potential users and the results of in¬ 
ternal testing and modification have contributed new concepts 
and indicators. In addition to the introduction of an "auto¬ 
mated" module, the analytic Early Warning and Monitoring 
System has experienced change and development. 


Recent project research memoranda reflect the current 
trends and developments. Aside from the automated alert 
scanning/historical search features of automated I&W, the 
recent enhancements and developments include the following: 
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• International political event/interaction data 
at the daily level (in addition to the monthly, 
quarterly, and yearly options) . 

• International political event/interaction data 
at the weekly level. 

• The addition to the basic system menu of "change 
input parameters." This option allows the 

user who is already in EWAMS to change the 
actors, flow, time parameters, time increment, 
data base, or all of the above. For example, if 
the user is looking at the USA and USR for February 
and March of 1979 (two-way) and wants to examine 
the USA and CHN for the same period, he would 
simply use the "change" option and enter the new 
pair of actors; the system would then automatically 
begin processing. 

• Text for regions is now available. 

• The user can now do runs for two special purpose 
regions (in addition to the existing capability 
of analyzing a special purpose region and another 
type of actor). 

• Additional special purpose region options are now 
available. The user can save and retrieve special 
purpose regions as well as edit them (add to/ 
delete from). In addition to the "create-your- 
own" special purpose region(s), the user can now 
access the country attribute typology-based 
regions; these groups, based on research conducted 
at the University of Maryland, are clusters of 
nations which statistically share various political, 
economic, military, and national capability charac¬ 
teristics. 

• In addition to the New York Times -based political 
event/interaction data, the user now also has 
access to comparable data coded at IPPRC from the 
Manchester Guardian . This alternative data base 
is especially useful for Western Europe and the 
Mediterranean area as well as sub-Saharan Africa, 
Eastern Europe, and South Asia. 

• All output is now labelled by source (NYT for New 
York Times and MAG for Manchester Guardian ). 

• Special Purpose Indicators (SPI's) are now avail¬ 
able to the user (for example, verbal versus 
physical conflict indicators or "create-your-own" 
special indicators). 
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• A single country performance indicator—Country 
Activity Profiles (CAP)—is now available to 
analysts. 

• The user may change source (NYT—►MAG) in all 
operational modules of automated I&W. 

The central components of the analytic EWAMS reflect a 
dual thrust which pervades IPPRC's efforts to: 

• Integrate and synthesize the best I&W-relevant 
research 

• Integrate and adapt this research in the form 
which is most relevant to the user community 

3.2.4 Dual source assessment . With the integration of 
NYT ( New York Times ) and MAG ( Manchester Guardian ) political 
event/interaction WEIS data into EWAMS, the user now has 
access to two separate data bases. The comparative performance 
of MAG and NYT has been assessed in retrospective and real¬ 
time modes. 

This research, which has been reported in Hopple (1979a, 
1979b, 1979c, 1980a, 1980c), has involved both the analytic and 
automated I&W components of EWAMS. Overall, the finding 
has been that MAG and NYT constitute two separate (albeit 
somewhat similar) "windows on the world." Although each 
has regional biases and strengths, the I&W analyst is advised 
in the typical case to use both data bases? this can be 
expected to provide a somewhat more reliable portrait of 
reality and minimize both "misses" and "false alarms." 
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In addition to WEIS, there are quite a few other events 
data sets in international and comparative politics. All 
are derived from one or more public newspapers or events 
data chronologies (e.g.. Facts on File , Deadline Data on World 
Affairs , etc.). The various events data collections are de¬ 
signed to map the domains of international and intranational 
affairs (primarily but not exclusively internation conflict 
and crisis and intranation violence and instability). 

Critics have been quite understandably skeptical of the 
practice cf using public source data for the purpose of 

monitoring and attempting to forecast internation behavior 
* 

patterns and crises. For reasons of cost and accessibility, 
however, the typical international relations events data 
project has relied on completely public sources. 

Ideally, an "on-line" intelligence-oriented inter¬ 
national affairs tracking and crisis warning system—such as 
EWAMS—would consist of an array of public and private/ 

* 

There has been some research on less open sources; the 
major example is the Foreign Relations Indicator Project 
(FRIP), which used the U.S. State Department's operational 
traffic as its source of data; see Burgess and Lawton (1972: 
49-51). Slater and Orloski (1978) relied on Foreign Broad¬ 
cast Information Service (FBIS) and cable traffic data in 
their research on Peruvian and Chilean intra- and inter¬ 
national affairs. 
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classified data bases. 


Such a system would impose a common 


data organization scheme on all types of incoming intelli¬ 
gence information, thus providing the intelligence analyst 
with a series of potential forecasts and estimates. Con¬ 
verging evidence would of course enhance the credibility of 
warnings; discrepancies could alert the analyst to the possi¬ 
bilities of "false alarms" (when one stream of data is in¬ 
correctly indicating a potential crisis) and "misses" (when one 
stream of data is ignoring a crisis detected by a second 
stream). 


Although public source data are suspect for a variety of 
reasons, several rather compelling arguments suggest that 
such data generally perform credibly and that much of the 
skepticism is unwarranted. The first factor is that most of 
the "dialogue" which constitutes interstate relations is 
actually captured by the elite media. To an extent, in 
fact, the prestige media structure the decisionmaker's (and 
everyone else's) image of the world (e.g., by highlighting 
issues and shaping issue agendas, by focusing attention or 
ignoring a given subject, etc.). 


Currently, a prototype study is being conducted by 
DARPA/CTD in the area of international and intranational 
event-oriented monitoring and warning systems with respect 
to sub-Saharan Africa (see, e.g., Hopple, 1979a). The effort 
is discussed below in Section II. IPPRC is amassing public 
source and FBIS data, while another contractor (Mathtech) is 
using an identical data collection scheme with cable traffic 
data. 
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It is well known, for example, that most intelligence 
data consist of publicly available information. It has been 
estimated that 90 to 95 percent of the data classified as "con¬ 
fidential" or above are from open sources. Not surprisingly, 
then, research on elite belief systems, which has relied ex¬ 
clusively on public source articulations (e.g., FBIS, New 
York Times , etc.) demonstrates that overtly stated elite be¬ 
liefs and values are frequently related to actual behavior in 
the sphere of foreign policy; the magnitude of the belief- 
behavior linkage is often robust. This finding holds for both 
open political systems and for non-polyarchic regimes; for 
details, see Hopple (1980b). 

The second reason for arguing that data derived from pub¬ 
lic sources like NYT and MAG are typically valid is the per¬ 
formance of the current version of the Early Warning and Moni¬ 
toring System (EWAMS). While EWAMS is far from infallible, 
real-time tests and the kinds of detailed source comparisons 
chronicled here and elsewhere (see various EWAMS memoranda) 
both demonstrate that EWAMS is a fairly viable (i.e., reliable 
and valid) system for: 

• Monitoring interstate interactions 

• Forecasting serious conflict and crisis 
episodes (albeit more impressively for cer¬ 
tain types of crisis)* 


Such as, those which escalate gradually (e.g., Iran-U.S. 
in 1979, Czechoslovakia-Soviet Union, the 1967 Middle East War) 
as opposed to crises which erupt suddenly (e.g., the 1973 
Middle East War). 
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The purpose or function of the EWAMS data collection 
operation suggests unequivocally that NYT should be supple¬ 
mented by a second, non-U.S. data source. The purpose of 
the data collection operations is twofold: to provide a 
portrait of the major events in the sphere of international 
relations (in retrospective and real-time modes) and to 
enable intelligence analysts in the U.S. to monitor and fore¬ 
cast international affairs and crises. The data base should 
therefore be reasonably comprehensive, given the interests 
and emphases of U.S. intelligence analysts . Presumably, 
the New York Times , the major U.S. "prestige newspaper," 
provides a good "map" of the world of international political 
event/interactions as it is perceived by the U.S. national 
security analysis and policy making communities. The purpose 
of the Manchester Guardian data base is to supplement the 
New York Times perspective and correct for its biases. 
Additionally, MAG and NYT together will presumably maximize 
"hits" and minimize "misses." Empirical results for the 
source comparisons are presented in Section 5.0. 

3.2.5 Software . FY 80 software development and en¬ 
hancement activities were unusually salient in nature and 
extensive in scope and magnitude. In addition to providing 
ongoing analytic programming support, the EWAMS computer 
science staff made numerous refinements and cosmetic enhance¬ 
ments to the EWAMS; also, several exhaustive reviews were 
conducted with CTD, leading to further changes. The 






enhancements to the master version of the EWAMS, which re¬ 
sides on the PDP 11/70 at DARPA/CTD's DDF, include keyword, 

SPI (Special Purpose Indicators), CAP (Country Activity Pro¬ 
file scores)—all new features of the analytic EWAMS—and 
the entire automated I&W module. The staff also converted, 
the EWAMS from an 11/70-based system written in FORTRAN to a 
Tektronix 4051/4907 file manager configuration written in 
BASIC; the latter was recently transferred to EUCOM/J-2 by 
CTD. Finally, work was recently completed which culminated 
in the release of a "new EWAMS" with new tails for indica¬ 
tors/probabilities, significantly reduced processing time, 
and an internally streamlined software structure. 

EWAMS has grown considerably since the beginning of 1979. 
At that time, the system consisted of what is now known as the 
"analytic EWAMS." The six main routines (crisis, change, 
tables, plots, text, and keyword)—together with a library 
of 50 to 60 subroutines—comprise the analytic software 
(see Figure 3-8). This initial core of software, forming 
the basic structure of the entire system, has remained largely 
intact with certain enhancements (see Pulley and Snyder, 

1979, for details). Figure 3-9 depicts the "current EWAMS." 

The EUCOM/EWAMS, designed to facilitate the tasks of 
EUCOM I&W analysts, is patterned after the PDP 11/70-based 
EWAMS but was modified and tailored to the needs of EUCOM 
and to the hardware configuration of the EUCOM/EWAMS. A 
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Figure 3-8 
ANALYTIC EWAMS 



User’s Guide (IPPRC, 1980b) and a Political Indicators Hand¬ 
book (IPPRC, 1980a) were developed to accompany the EUCOM/ 
EWAMS, which was recently transferred. A computer-based In¬ 
struction Aid Package was also programmed; a System Flowchart 
provides an overview of the software base of the EUCOM/EWAMS. 

Figure 3-10 depicts the EUCOM/EWAMS "architecture." The 
system features two major components: text and crisis. The 
text component permits the user to retrieve actual text items; 
the crisis component offers four system options. In crisis, 
the user must enter input information (e.g., what countries 
he wants to look at and for what period) and may then have. 
the results displayed in tabular or graphic (plots) form; 
he may also examine text or change the input information in 
order to conduct another analysis (e.g., study a different 
country pair for the same time period, analyze the same coun¬ 
tries for a different aggregation and/or time frame, etc.). 
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Figure 3-10 

THE STRUCTURE OF THE EUCOM/EWAMS SYSTEM 
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The EUCOM/EWAMS utilizes a Tektronix 4051 graphics 
terminal/microprocessor and a 4907 File Manager. To activate 
and use the system, the user must insert the appropriate 
diskettes into the 4907. For runs involving the crisis 
module, the software and data base diskettes must be inserted; 
for runs in which the analyst wants only textual output, the 
software and text diskettes must be inserted. A tape must 
also be inserted into the 4051. These steps are all covered 
in the User's Guide (IPPRC, 1980b); the 4051-based Instruction 
Aid Package also reviews the use of the 4051. 
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4.0 PROBABILISTIC FORECASTING 


The Early Warning and Monitoring System (EWAMS) provides 
estimates of the probability of a crisis during any month for 
any nation pair. Such probabilities aid in the task of crisis 
warning by recommending "crisis" or "no crisis" forecasts 
while recognizing (and taking into account) a margin of error. 
The initial procedures for estimating EWAMS probabilities 
relied upon labor-intensive research which was limited to 
estimates based upon a single indicator. An alternative pro¬ 
cedure—discriminant analysis—may be automated and may process 
multiple indicators to estimate crisis probabilities. 

The numerous difficulties associated with providing warn¬ 
ing of international crises are no more troublesome than a 
large variety of forecasting problems. However, international 
crises, which are unique and specific events, require fore¬ 
casting methods which are compatible with the discrete nature 
of the subject matter. The "yes/no” dichotomy approach to 
crisis forecasting reduces the warning task to a simple 
statement of certitude, an absolute prediction within a world 
of uncertainty, margins of error, and experiential relativity. 

Nevertheless, meteorologists who seek to forecast 
thunderstorms, tax auditors who hope to uncover false reports, 

* 

This section draws primarily on Rossa (1979c). 
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and many other professionals must provide predictions of a 
dichotomous nature on a daily basis. Indeed, many policy 
questions revolve around the occurrence (or non-occurrence) 
of a dichotomously-phrased result: will it happen or will 
it not? Crisis forecasting must be defined in similar terms. 

The absolute quality of crisis warning can be compromised 
in order to reflect more accurately the inadequacies of our 
forecasting capability. Whereas the appearance of a crisis 
has an objective probability of either unity or zero, the 
forecaster who lacks a perfect and deterministic model must 
"hedge" his or her predictions by admitting a margin of error. 
This is accomplished by producing probabilistic forecasts, 
which state clearly the likelihood of the event as approaching 
zero or one. Certainty is made relative by probabilities and 
their distances from absolutes; where uncertainty exists, it 
is expressed as a measurable quantity. While such forecasts 
often appear to be cowardly or removed from reality (e.g., a 
probability of precipitation of .3 or .6 is reported during a 
rainstorm), they represent an improvement over guesswork to 
the extent that they correctly and accurately identify the 
degree of prediction capability. 

The essential goals of probabilistic crisis forecasting 
are: (1) a device which improves upon guesswork by providing 

an accurate likelihood estimate which varies on the basis of 
available information; (2) a device which.permits dichotomous 
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choices—"crisis" versus "no crisis" forecasts—and improves 
upon the success of guesswork. In crisis forecasting, the 
term "guesswork" implies the rational decision to never fore- j 

cast an international crisis; not only are crises infrequent, j 

since they have a very low probability in the absence of in¬ 
formation at any particular time in space, but they are also 
bureaucratically dangerous forecasts when false alarms result 
(e.g., Chan, 1979). 

The use of information to provide probabilistic crisis 
forecasts is beneficial to the extent that the resulting 
statements of the form; 

"Y out of 100 times, a crisis occurs when informa¬ 
tion says X," 

are accurate over numerous observations, are variant across 
different information matrices X, and are polar such that 
Y tends to be 0 or 100. Accuracy is essential if the proba¬ 
bilities are to be trusted, but accuracy alone is insuffi¬ 
cient; information can be completely useless or may be ig¬ 
nored—yet there is accuracy in the sense of a simple, low 
probability of crisis. Insisting upon variation across dif¬ 
ferent information matrices results in accurate statements 
and probabilities which change as information changes. This 
constitutes a demand for a relationship between probabilities 
and information; the requirement of polarity concerns the 
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strength of the relationship and recognizes that the stronger 
(or more deterministic) the informational context regarding 
crisis likelihoods, the closer to absolute certainty (and 
utility) are the relevant forecasts. 

Regarding the goal of dichotomous choices of "crisis" 
or "no crisis" forecasts, probabilistic forecasts which 
retain the three aforementioned characteristics to an 
acceptable degree can be entrusted to a straightforward 
decision-rule: whenever the probability of crisis is 
greater than one-half, a "crisis" forecast is rational; 
otherwise, a "no crisis" prediction is warranted.* The 
reasonableness of this rule is based upon the premise that 
the decisionmaker wishes to be correct more often than wrong. 
Whenever a crisis probability of over one-half is generated, 
a decisionmaker will be correct over one-half of the time and 
more by selecting the "crisis" forecast. Indeed, the 
expected rate of success is expressed as a percentage by 
multiplying the probability estimate by one hundred: a .6 
probability equals a 60 percent success rate for crisis fore¬ 
casts. 

The dichotomous nature of the forecast subject—crises— 
and the interval nature of the information—indicators— 

This rule can be modified by weighting probabilities 
by utilities or disutilities if "false alarms" or "misses" 
are differentially costly. See Morrison (1974: 454). 










recommends a reliance upon discriminant functions for 
probability estimation. Discriminant functions estimate the 
probability that a case belongs to a given category, such as 
"crisis" or "non-crisis" categories, on the basis of indica¬ 
tors. Discriminant analysis is the process of generating 
probabilistic estimates and classifications (using the .5 
probability criterion) which maximize accuracy, polarity, 
and variation. 

* 

4.1 The Discriminant Analysis Methodology 


Figure 4-1 depicts the application of discriminant 
methodology within the EWAMS. The methodology can contribute 
to the "analytic" component by offering an internalized proce¬ 
dure for generating probabilities for current and future 
indicators, as well as for multiple-indicator information 
matrices. Discriminant methodology can also contribute to the 
automated EWAMS by offering a research tool with which to 
increase the performance of automated scan "advice" output 
and enhance the flexibility of directed scans. Machine-inten¬ 
sive methods thus aid and replace labor-intensive tasks. 


This section is based upon Rossa (1979b: 20-23). Cer¬ 

tain liberties are taken in order to provide a non-technical 
description of the analytic technique. One of these involves 
an assumption of a dichotomous "dependent" variable, although 
N-category variables are permissible (e.g., "crisis," "no 
crisis," "pre-crisis," "post-crisis"). Another assumption is 
linearity; although non-linear models can be tested, the 
assumption is that this would result only from research find¬ 
ings and recommendations which point in that direction. 
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Figure 4-1 


DISCRIMINANT ANALYSIS METHODOLOGY 
IN THE EARLY WARNING AND MONITORING SYSTEM 
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Discriminant analysis combines ideas from classical and 
Bayesian statistics in order to estimate the probability that 
a given case is or is not of a certain type. The "type" of 
case with which we are concerned is the crisis case; our cases 
are observations of a dyad at several time-points. Thus, dis¬ 
criminant analysis enables us to estimate the probability that 
a dyad at time t is a crisis case, or, equivalently, the 
probability of a crisis for the dyad at time t. The single 
indicator crisis probability is estimated through a series of 
steps. Using retrospective (known) data, "crisis" cases are 
separated from "non-crisis" cases to form two samples. The 
mean or "average" crisis case indicator reading is found, along 
with the average non-crisis case indicator level. Each mean 
will have an associated standard deviation, of course, and 
we shall assume normal distributions about the means. For 

each mean, we can calculate the probability of observing "x" 

* * * 

where "x" is an indicator reading. 


* 

Assume that there is no lag for simplicity in wording. 
With a lag of one increment, we "estimate the probability that 
a dyad at time t is one increment prior to a crisis, or, 
equivalently, the probability of a crisis for the dyad at time 
t + 1." With a lag, we discriminate not between "crisis" and 
"no crisis" time intervals, but between time-increments which 
are "N-lags prior to crisis" and time increments which are not. 
** 

Multiple-indicator crisis probabilities are estimated in 
the same manner after they are reduced to a single "factor" or 
dimension, defined by the OLS multiple regression equation; 
e.g., they are weighted to provide a single estimate of the 
dependent variable and these estimates are treated as one 
variable. 

Technically, "x" is "any value as far, or further, from 
the mean as the indicator reading." The idea would not be 
unfamiliar to those who use significance tests. 
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For each case we may calculate: 


• P(x | c) 

• P(x J n) 

where "x" is its indicator reading, "c" is the mean (and 
standard deviation) of the sample of crisis cases, and "n" is 
the mean (and standard deviation) of non-crisis cases. Note 
that the two probabilities need not add to one; they will do 

so only to the extent that the indicator is a perfect dis- 

★ 

criminator (i.e., that is has a perfect threshold value). 

The calculations translate into: 

• The probability of x if this is a crisis case 

• The probability of x if this is a non-crisis case 


These two probabilities are useful because they are re¬ 
quired to apply Bayes' Theorem to the estimation problem. 
Bayes' Theorem states that: 


p, e | _ P(c) * P(x I c) _ 

P(c| x) P(c) * P(x | c) + P(n) 


The term P(c | x) is the probability that the case is a "cri¬ 
sis," given what the indicator tells us (x). Two of the other 


* 

Actually, this holds to the extent that the relationship 
between indicator and crisis probability is deterministic, 
with no residual error. This is more strict than "a perfect 
threshold." 
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four different terms in the Bayesian formula are supplied 
above through classical statistics; the others, P(c) and P(n), 
are the a priori estimates of the likelihood of crisis or 
non-crisis, or the "priors," and add to one. "Priors" may 
be set equal, set to the proportion of cases in each category, 
or set by the analyst himself. 


Note that the only information required from classical 
statistics, and therefore from retrospective data, for the 
calculation of probabilities is the set of means and standard 
deviations which defines "c" and "n".** Given this informa¬ 
tion, any dyad for any time period can be assigned a crisis 
probability based on the indicator reading. The formula is 
established for all cases not included in retrospective data; 
the only required information is the new indicator value and 
the result is a crisis probability estimate. 


Thresholds are also defined. As discussed earlier, fore- 
casts are most efficient, by definition, when a case is placed 


This last option might be offered to the user of the 
EWAMS. His or her own a priori estimate of a crisis could 
be obtained by asking for an estimate for the dyad and time 
frame which is the focus of concern. Although this is not a 
"true" a priori and many "pure" theorists would reject the 
approach as one which introduces bias, the bias operates to 
revise probability estimates toward the a priori of the 
analyst; this can be interpreted as adding user knowledge. 

In any event, miss and false alarm rates would signal the 
biasing effect to the analyst. 

** 

Actually, two formulae are obtained; they provide, for 
each category (crisis, non-crisis), the z-score of the "pre¬ 
dicted dependent variable." In this manner, multivariate 
situations are covered. 
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in a category if the probability of a correct categorization 
is .50 or above. Thus, if P(c | x) = .50, then indicator level 
"x" forecasts a crisis. Where P(c J x) = .50, "x" is the 
threshold value. Once categorizations are made for retro¬ 
spective cases, miss/false alarm rates are reported; these can 
be supplied whenever the indicator is utilized in order to 
facilitate judgments regarding probability estimates for new 
cases. 

Discriminant analysis is an analytical technique which 
provides a method of estimating the (most likely) probability 
of crisis, given the selected indicator(s), for a dyad at a 
certain time. It uses retrospective data to establish a 
formula for the estimate and requires a Bayesian prior; any 
new indicator reading will produce a probability estimate. A 
forecast of a crisis is warranted if the estimated probability 
is one-half or better, and miss/false alarm rates for the 
formula are based on the retrospective cases. The methodology 
can be utilized to establish leads/lags and automated I&W alert 
list generators and can be incorporated into the EWAMS for 
probability estimates derived from user-supplied specifications. 

4.2 Monthly Probabilities 

Prerequisite to tests of the discriminant analysis 
methodology for probabilistic crisis forecasts is the creation 
of a retrospective data base. This data base must constitute 
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a substantial sample of crisis cases and non-crisis cases 
(all characterized by indicators) to be employed in the 
EWAMS during real-time crisis monitoring and warning. The 
essential task of the discriminant analysis method is the 
identification of crisis cases within this sample through 
probability estimates derived from these indicators. 

Crises are relatively unusual situations in interstate 
relations. If we define a case as an interstate dyad during 
a specific time period, tens of thousands of cases are avail¬ 
able on each day, week, or month. Yet, the percentage of 
these which may be labeled "crisis" cases is quite low. 

The selection of a sample of crisis and non-crisis cases 
must begin with the identification of the infrequent crisis 
cases in order to ensure the inclusion of a sufficient number 
of such cases; a simple random sample of dyads at various 
points in time is likely to exclude all crises. However, the 
task of defining crisis cases is both theoretically and oper¬ 
ationally difficult; not only do numerous definitions of 
"crisis" pervade the literature of international political 
theory (see, e.g., Hopple and Rossa, 1980), but empirical 
crisis "lists" have also proliferated with a disconcerting 
lack of agreement (see, e.g., Thompson, 1979). 

The overriding consideration in selecting a crisis inven¬ 
tory is the source of information which will permit analyses: 
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the EWAMS indicators drawn from a WEIS data set which includes 
all countries for all time periods since January 1, 1966. All 
crisis and non-crisis cases which enter the retrospective data 
base sample must fall within this time frame in order to uti¬ 
lize the indicators. Therefore, crises which occurred prior 
to 1966 must be excluded. Moreover, recent crises should be 
included in order to maximize the current relevance of the 
methodology; crises as late as 1978 are desirable. Finally, 
our crisis inventory should be as large as possible, covering 
all regions of the world (but excluding cases where the "cri¬ 
sis" label is likely to be used only when a particularistic 
set of criteria is employed). 

These criteria focus on the requirements of crisis selec¬ 
tion and do not concern the more difficult task of crisis 
dating . Theoretical and operational definitions often dis¬ 
agree regarding the time at which a crisis took place, even 
when agreement exists that a crisis did occur. The disagree¬ 
ments often concern the month in which the crisis began, and 
are almost inevitable in attempts to identify the initial day 
of a crisis; similar discrepancies arise regarding the end- 
date of a crisis. Because the Early Warning and Monitoring 
System seeks to warn of crises, through probabilistic fore¬ 
casts, at levels of time aggregation as low as daily, the 
start date (day) of the crisis is the crucial parameter in a 
inventory. 
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Six available inventories of crises were studied in an 


attempt to isolate cases for the discriminant analysis tests. 
First, Daly and Bell (1977) of the Early Warning and Monitor¬ 
ing Project provide an inventory of fourteen crises, encom¬ 
passing 27 dyads; this serves as the foundation for our 
analysis. This inventory meets the condition of excluding 
crises which occurred prior to 1966. The inventory cannot 
meet the requirements of recency (it includes crises through 
1974) or of a daily start date (it employs monthly aggrega¬ 
tions) . 


The Mcore et al. (1975) report was the basis of the 
Daly/Bell inventory and provides start dates which are spe¬ 
cific to the day. This list includes crises between 1946 and 
1974 (inclusive) and six crises during the 1966-1974 period 

which were deleted by Daly/Bell because of lack of sufficient 
* 

data. However, the Moore inventory fails to cover the post- 
1974 period, and has been criticized because of its opera¬ 
tional definition and dating procedures (Thompson, 1979). 


* 

This exclusion is based upon the usually correct tradi¬ 
tion in social statistics of excluding cases with missing 
data. However, it could be argued that this tradition should 
not apply to events data research. Unless events are 
purposely excluded from the data set, we must assume that the 
number of events is a meaningful "zero" when none is reported. 
To claim a situation of "missing data" and exclude a case 
results in the logically unwarranted claim that numerous 
events have not been reported, even when some events are 
reported, and all event totals are incomplete. In any case, 
data base inadequacies will affect real time performance and, 
perhaps, should be anticipated in retrospective analyses. 
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The Daly/Bell inventory adds one case which was not in the 
Moore report. 

The Thompson (1979) report attempts to improve upon the 
Moore and Daly/Bell inventories by devising a more acceptable 
set of operational criteria to delineate and date crises and 
by updating the inventory through 1977. Whereas Moore et al. 
(and Daly/Bell) utilize an "interaction" level of analysis in 
defining crises as significant departures from normal ex¬ 
changes, Thompson argues in favor of a "decisionmaking" 
level of analysis for definitional purposes. As a result, 
Thompson rejects five of the fourteen Daly/Bell crises drawn 
from Moore, alters the start date in another three cases, and 
agrees with only six of Moore's nineteen crisis/dates during 
the period. This is not unusual, given the discrepant levels 
of analysis of the two inventory operationalizations. Thompson 
adds eleven crises to the combined inventory, nine of which 
began after 1974. Nevertheless, the interaction level of 
crisis analysis is more appropriate to the event/interaction 
data utilized by the EWAMS and the Thompson report is valuable 
only insofar as additional crises are identified; the Thompson 
dates and rejections of crisis cases are based upon a more 
circumscribed and inappropriate level of observation than 
that required by the EWAMS. 

Two reports by researchers who focus upon a specific 
nation's decisional apparatus are relevant to a crisis 
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inventory: Hazelwood and Hayes (1976), which treats U.S. 
crises, and Mahoney (1978) , which concerns Soviet crises in 
foreign affairs. These inventories are based upon decision¬ 
making definitional criteria, as is Thompson's, but are 
specific to one decision unit. They are therefore even 
less applicable to an interaction-oriented analysis. However, 
their value in dating crises which have been identified by 
more cross-national or international definitional criteria 
cannot be dismissed. 

Finally, Butterworth (1976) offers an inventory which 
includes conflicts , as opposed to crises, which are managed 
by interstate agents. Butterworth's inventory is useful for 
our purposes because it is more general, featuring many 
situations which may represent crises, and narrative, thereby 
aiding the dating process. It cannot be used alone because 
of its lack of comprehensiveness (unmanaged crises and post- 
1974 crises are not included) and exclusiveness. 

The shortcomings of each extant crisis inventory obviate 
a reliance upon a single work. An integrative strategy is 
therefore necessary. Table 4-1 provides an integrated 
crisis inventory, including dyads and start dates, and sum¬ 
marizes the position of each source regarding these. From 
1966 through 1974 (the period covered by Moore) twenty crises 
are included; one of Moore's nineteen crises was rejected 




Table 4-1 


CRISIS LISTS: FORTY-EIGHT CRISIS DYADS, 1966-1978 


Dyads Start Date 

1. SYR-JOR 661201 
PLO-JOR 


2. USR-CHN 670111 


Sources 

Moore, Daly/Bell. 
Not in Butter- 
worth, CACI, 
Mahoney. 

Thompson rejects. 

Moore, Daly/Bell, 
CACI. Mahoney: 
670126. Not in 
Butterworth. 
Thompson rejects. 


Comments 

Start date some¬ 
what arbitrary, 
but good appoxi- 
mation. 


Start date some¬ 
what arbitrary, 
but good approxi¬ 
mation. First 
massive demonstra¬ 
tions: 670126. 


3. ISR-SYR 670605 
ISR-UAR 
ISR-JOR 


Daly/Bell, 
Mahoney, Butter- 
worth. Moore, 
Thompson , CACI: 
670514. 


Earlier start date 
for alerts; later 
start date for war. 


4. UNK-CHN 670627 


Moore. Not in 
Daly/Bell, CACI, 
Thompson, Mahoney, 
Butterworth. 


Start date for Hong 
Kong dispute; first 
military clash on 
670804. 


5. GRC-TUR 671115 
GRC-CYT 
TUR-CYP 


6. ISR-JOR 680102 


7. USA-KON 680123 


8. USR-CZE 680820 


Daly/Bell, Moore, Start date for 
Thompson. Not in dispute. 
Butterworth, CACI, 

Mahoney. 


Moore, Daly/Bell. 
Mahoney: 671121. 
Thompson rejects. 
Not in Butter¬ 
worth, CACI. 

Moore, Daly/Bell, 
CACI, Butter¬ 
worth, Mahoney, 
Thompson. 

Daly/Bell, 

Butterworth. 

Not in Mahoney. 
Moore, CACI: 
680716. Thompson: 
680509. 


Start date for 
initial fighting. 
Major clashes on 
680125, 680321. 


Start date for 
Pueblo seizure, 


Start date for 
invasion. 
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Table 4-1 


1 


CRISIS LISTS: FORTY-EIGHT CRISIS DYADS, 1966-1978 (Cont'd.) 



Dyads 

Start Date 

Sources 

Comments 

1 9- 

ISR-UAR 

ISR-SYR 

690244 

Moore, Daly/Bell, 
Mahoney, Butter- 
worth. Not in 

CACI. Thompson 
rejects. 

Start date for 
emergency declara¬ 
tion and border 
raid. War of 
Attrition one month 
later. 

10. 

USR-CHN 

690302 

Moore, Daly/Bell, 
Mahoney, Butter- 
worth , Thompson. 

Not in CACI. 

Start date of 
military clash. 

11. 

HON-ELS 

690630 

Moore, Butter- 
worth. Not in 
Daly/Bell, CACI, 
Thompson, 

Mahoney. 

Start date for 
troop concentra¬ 
tions. 

12. 

VTN-USA 

VTN-CAM 

VTN-VTS 

700318 

Thompson, Butter- 
worth. Not in 
Daly/Bell, CACI, 
Mahoney, Moore. 

Start date of coup. 
700429, 700501 US, 
VTS invasion. 

13. 

UGA-TAZ 

700709 

Moore, Butter- 
worth. Not in 
Daly/Bell, Thompson 
CACI, Mahoney. 

Start date of 
post-coup fighting. 

14. 

JOR-SYR 

JOR-PLO 

JOR-IRQ 

USA-SYR 

USA-USR 

ISR-SYR 

700901 

Daly/Bell: no 

Iraq. Thompson: 
no PLO. Moore: 
no PLO, Iraq. 
Mahoney: 700920. 
CACI, Butterworth. 

Start date of first 
military, diplomatic 
fighting. USA, ISR, 
SYR actions later in 
month. 

15. 

IND-PAK 

711121 

Butterworth, Daly/ 
Bell, CACI. 

Moore: 711127. 

Thompson: 710327. 
Mahoney: 710423. 

Start date of in¬ 
vasion. Earlier 
dates closer to 
civil war start 
date. 

16. 

UGA-TAZ 

720917 

Moore, Butter¬ 
worth. Not in 
Daly/Bell, CACI, 
Thompson, Mahoney. 

Start date of 

Obote attack. 
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Table 4-1 


CRISIS LISTS: FORTY-EIGHT CRISIS DYADS, 1966-1978 (Cont'd.) 


Dyads Start Date 
17. RHO-ZAM 730109 


18. ISR-UAR 731006 
ISR-SYR 


19. USA-USR 731024 


20. GRC-TUR 740715 
CYP-CYT 

21. USA-CAM 750512 

22. MOR-SPN 750527 

23. MOR-ALG 751209 


24. SYR-LEB 760601 
SYR-PLO 
SYR-ISR 


Sources 

Moore, Butterworth. 
Not in Daly/Bell, 
CACI, Thompson, 
Mahoney. 

CACI, Butterworth, 
Daly/Bell. Moore: 
731001. Mahoney 
and Thompson: 
731003. 

Butterworth, Daly/ 
Bell. CACI: 

731021. Moore: 
731007. Mahoney: 
731003. Thompson 
rejects. 

Mahoney, Daly/Bell, 
CACI, Butterworth. 
Not in Moore. 
Thompson: 740703. 

Thompson. Not in 
Daly/Bell, Moore, 
CACI, Mahoney, 
Butterworth. 

Thompson: 750428. 

Not in Daly/Bell, 
Moore, Butterworth, 
CACI, Mahoney. 

Thompson: 751015. 

Not in Daly/Bell, 
Moore, Butter¬ 
worth, CACI, 

Mahoney. 

Thompson: 760106. 

Not in Daly/Bell, 
Moore, Butterworth, 
CACI, Mahoney. 


Comments 

Start date of 
border closing 
after Rhodesian 
attacks. 

Start date of 
military attack. 
Early start dates 
for military 
preparations. 

Start date of 
global alert. 
Earlier start 
dates for Mideast 
War, embargo. 


Start date of coup, 
civil war. Turkey 
invasion 740730. 


Start date of 
Mayaguez seizure. 


Start date of war 
alert status. 


Start date of war 
preparations. 


Start date of 
invasion. 
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Table 4-1 


CRISIS LISTS: FORTY-EIGHT CRISIS DYADS, 1966- 


Dyads Start Date 

25. ANG-COP 770310 

26. UAR-LBY 770719 

27. SOM-ETH 770808 

28. ISR-SYR 770919 
ISR-PLO 

29. UGA-TAZ 781031 

30. CAM-VTN 781228 


Sources 


Not in Thompson, 
Daly/Bell, Moore, 
Butterworth, CACI, 
Mahoney. 

Thompson: 770326. 
Not in Daly/Bell, 
Moore, Butterworth, 
CACI, Mahoney. 

Thompson: 770222. 
Not in Daly/Bell, 
Moore, Butterworth, 
CACI, Mahoney. 

Thompson: 770602. 
Not in Daly/Bell, 
Moore, Butterworth, 
CACI, Mahoney. 

Not in Thompson, 
Daly/Bell, Moore, 
Butterworth, CACI, 
Mahoney. 

Not in Thompson, 
Daly/Bell, Moore, 
Butterworth, CACI, 
Mahoney. 


•1978 (Cont'd.) 


Comments 

Start date of in¬ 
vasion of Shaba. 


Start date of 
military conflict. 


Start date of w - 
situation. 


Start date (approxi¬ 
mate) of invasion. 


Start date of 
Ugandan invasion. 


Start date of initial 
armed fighting in 
invasion. 
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Table 4-1 (Cont'd.) 



I 


SIX REJECTED CRISIS DYADS, 1966-1978 


Dyads Start Date 
UNK-RHO 660405 

ISR-LEB 670605 

USA-KON 690415 

ISR-SYR 740822 

USA-KON 760818 

VTN-CHN 790217 


Sources 

Moore. Not in 
Butterworth, Daly/ 
Bell, CACI, 

Thompson, Mahoney. 

Moore. Not in 
Butterworth, Daly/ 
Bell, CACI, 

Thompson, Mahoney. 

Thompson. Not in 
Daly/Bell, Butter¬ 
worth, CACI, 
Mahoney, Moore. 

Thompson rejects. 
Not in Daly/Bell, 
Butterworth, Moore, 
Mahoney, CACI. 

Thompson. Not in 
Daly/Bell, Butter¬ 
worth, CACI, 
Mahoney, Moore. 


Not in Thompson, 
CACI, Butterworth, 
Daly/Bell, Mahoney, 
Moore. 


Comments 

Blockade start 
date. Too early 
for tests. 


Lebanon is 
peripheral actor. 


Plane downed, very 
minor incident on 
evidence. 


No crisis recom¬ 
mendation. 


Start date of 
tree-trimming in¬ 
cident, ques¬ 
tionable threat 
of war. 

Start date of 
war, past cut-off 
date. 
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(for reasons to become apparent); two not listed by Moore 
were added on the basis of evidence drawn from other sources 
including the one added by Daly/Bell. Of the eighteen Moore 
crises, twelve are unchanged in Table 4-1. Five changes in 
the start dates are made: 


• For the 1967 Middle East war, the start date for 
military engagements is chosen over the start 
date of military alerts. This is in accordance 
with Daly/Bell, Butterworth, and Mahoney, and 
represents a more definitive change in inter¬ 
actions. 

• For the 1968 Soviet-Czechoslovakia crisis, the 
start date for the invasion is chosen over 
earlier estimates because of a lack of con¬ 
sensus regarding other dates and in accordance 
with Daly/Bell and Butterworth. 

• For the 1971 Indo-Pakistani War, the start 
date is six days prior to Moore's estimate 
(as recommended by CACI and Butterworth). 

• For the 1973 Middle East war, the start date 
for military attacks is selected over earlier 
preparatory actions in accordance with CACI 
and Butterworth. 

• For the U.S. Global Alert of 1973, the date 
of the alert is chosen over a variety of 
earlier alternatives. 


In one other case, dyads chosen by Moore for a crisis are 
changed; Iraq and the PLO are excluded by Moore from the 
1970 Middle East crisis, but most accounts include one or 
both actors as central participants in the crisis. 

While Moore and Daly/Bell isolate crises before 1975 


and other sources are used to adjust the inventory, no 








source satisfactorily covers the 1975-1978 period. Only 
Thompson's nine post-1974 crises are available, and these are 
based upon decisionmaking definitional criteria. Therefore, 
Thompson's list has been modified after extensive analysis of 
newspaper reports, as well as the Thompson documentation, 
utilizing the interaction level of crisis definition. As a 
result, seven of the post-1974 Thompson crises are included 
in Table 4-1, although six start dates are adjusted. One 
"crisis" is rejected by Thompson and is also excluded here; 
another (the tree-trimming incident in the Korean demilita¬ 
rized zone) is rejected (despite Thompson's recommendation) be¬ 
cause of a lack of evidence of interactional change. One cri¬ 
sis during the period covered by Thompson is added to his 
list: the Katanga rebel invasion of Zaire. We identify two 
1978 crises: the Tanzanian invasion of Uganda and the Viet¬ 
namese invasion of Cambodia. 

The forty-eight crisis dyads in Table 4-1 define the 
sample of cases which enters the following tests of dis¬ 
criminant analysis methodology. This analysis deals exclu¬ 
sively with monthly time aggregations; the target of proba¬ 
bilistic forecasts is the crisis-month defined as the month 
in which the crisis start date falls. These crisis cases 
are contrasted with non-crisis cases, using EWAMS indicators 
and predictors of case-type. 
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4.2.1 Cases and variables . The availability of a 
crisis inventory allows us to construct a data set with which 
the discriminant methodology may be tested. This data set is 
composed of (1) cases and (2) variables. These must be chosen 
carefully in order to test accurately the probabilistic crisis 
forecasting power of any methodology. 

The cases which comprise the data set are of three genres 

• Crisis months (as defined in the previous 
section) 

• Precrisis months, which are those that 
occur prior to the crisis and which may 
permit crisis forecasts with a lead/lag 

• Noncrisis months, which contain no crisis 
start date and do not immediately pre¬ 
cede a crisis 

For each crisis within our inventory, these cases are defined 
as depicted in Figure 4-2: two precrisis months are specified 
and correspond to the two months immediately preceding the 
occurrence of the crisis; nine noncrisis cases are defined in 
each case and include the nine months prior to the precrisis 
cases. In this manner, twelve months of data are included in 
the sample of cases for each of the forty-eight dyads in the 
inventory; the total is thus 576 cases. 

The specification of two precrisis months is a procedure 
which facilitates testing lead/lag relationships in probabil¬ 
istic forecasts; in this study, a two-increment/month lead/lag 
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Figure 4-2 


THE TEST CASES 



1 2 3 4 5 6 7 8 9 10 11 12 

MONTHS 


is the maximum for testing purposes. The selection of nine 
preceding months for the sample of noncrisis cases ensures 
that: 


• Crisis months represent relatively infrequent 
cases in our sample, corresponding to actual 
frequencies of observation. 

• Noncrisis months are close to the occurrence 
of crises in order to maximize the level of 
difficulty or subtlety in discrimination. 

• Noncrisis months do not represent postcrisis 
periods and rely upon estimates of crisis 
end-dates. 


The requirement of eleven months of data prior to a crisis 
obviates the inclusion of the Rhodesian Blockade case, which 
occurred prior to December 1, 1966. 


The variables which characterize this data set are 
drawn from the current Early Warning and Monitoring System, 
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which offers ten indicators. Unusualness (or standardized) 


scores of an indicator rely upon an historical reference, or 
"tail," to which current raw scores are compared; in this 

study, unusualness is measured by reference to the 5-year 

** 

(60-increment) period preceding the crisis month. 

The average values of indicators in our sample and the 
standard deviations are reported in Table 4-2. The figures 
per se are revealing and anticipate the later findings. The 
descriptive data patterns demonstrate the vast differences 
among the subclasses of months within the data base. Crisis 
months in particular score unusually high, on the average , 
for every indicator . The total number of events in the mean 
crisis month is nearly seven times the mean for noncrisis 
months, and three-to-five times the average for precrisis 
readings. These results are most marked for conflict events 
and are reflected in measures of variety. Furthermore, with 
the exception of the cooperation Z-score, every indicator in¬ 
creases in value as the (average) crisis approaches; noncrisis 
months have the lowest scores, and precrisis I cases have the 
highest readings aside from crisis months. These average 
values suggest that the indicators increase as a crisis 

* 

See Daly and Davies (1978: 25-30) and Section 3.1.1 

above. 

The 60-increment tail is now used for monthly and all 
other time aggregations in EWAMS; see Rossa (1979a) for the 
relevant empirical research on alternative historical tail- 
lengths. 
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Table 4-2 


MEANS AND STANDARD DEVIATIONS 


MEANS 

CRISIS PRE-CRISIS I PRE-CRISIS II NO CRISIS ALL 

(n = 48) (n = 48) (n = 48) (n = 432) (n = 576) 

TOTAL ACTIVITY 27.27 7.56 5.33 3.99 6.34 

TOTAL Z-SCORE 8.58 2.19 0.90 0.39 1.27 

COOPERATION 6.63 2.04 1.73 1.39 1.91 

COOP. Z-SCORE 4.37 0.47 0.66 0.21 0.62 

CONFLICT 20.67 5.52 3.60 2.60 4.43 

CONFLICT Z-SCORE 10.33 3.14 0.49 0.35 1.42 

TENSION 63.96 40.92 29.10 23.26 28.61 

TENSION Z-SCORE 2.75 1.79 1.48 0.20 0.57 

HREL 0.42 0.23 0.17 0.13 0.17 

HREL Z-SCORE 3.43 1.29 0.29 0.25 0.60 

STANDARD DEVIATIONS 

TOTAL ACTIVITY 37.73 12.52 8.69 7.64 13.85 

TOTAL Z-SCORE 13.41 5.21 0.39 2.27 4.65 

COOPERATION 13.45 6.29 3.36 4.87 6.07 

COOP. Z-SCORE 7.91 1.87 2.66 1.64 2.83 

CONFLICT 29.31 8.40 6.21 5.91 10.31 

CONF. Z-SCORE 18.12 6.59 1.75 2.38 5.96 

TENSION 28.54 33.12 32.63 30.30 30.95 

TENS. Z-SCORE 2.84 3.41 1.48 1.52 1.73 

HREL 0.23 0.24 0.21 0.20 0.21 

HREL Z-SCORE 4.63 2.39 1.29 1.34 1.94 


approaches and that crisis and precrisis months can be dis¬ 
tinguished unambiguously from noncrisis cases. 


The standardization of indicators describes the degree of 
heterogeneity of cases within the sample. Large standard 
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I 

1 

deviations, which reveal heterogeneity, may render differences 
in means statistically insignificant in discriminant testing. 
Heterogeneity on all volume indicators is very high for crisis 
months and is also high in precrisis I months, as compared to 
noncrisis cases. Precrisis II cases are generally not more, 
or less, homogeneous than noncrisis months, but are nearly 
invariant on the total activity unusualness score (s = .39). 
Tension and uncertainty scores vary without regard to the 
classification scheme, but variety unusualness indicators are 
least homogeneous for crisis and precrisis I months. 

These descriptive findings can be summarized as follows: 

• Indicators of event volume and variety generally 
increase as a crisis approaches. 

• Indicators of event volume and variety become 
more varied as a crisis approaches, although 
there are exceptions. 

In anticipation of the discriminant analysis results, these 
findings portend both failures and successes in probabilistic 
forecasts. The average differences across crisis, precrisis, 
and noncrisis cases suggest meaningful contrasts and accurate 
discriminations. However, the heterogeneous nature of crisis 
(and precrisis) indicator readings will undoubtedly result in 
statistically insignificant differences and/or cases of pre¬ 
dictive error. An analysis of current capabilities will pre¬ 
cede a discussion of the results based on the discriminant 
methodology. 



The initial 


EWAMS probabilities were single-indicator in nature and at 
the ordinal level of measurement. Although the process of 
calculating crisis probabilities in this manner is tedious 
and inefficient when large numbers of indicators are available 
(Rossa, 1979b), a methodology which supplants this approach 
should equal or exceed the success rate of the established 
record. This record is, however, incomplete: serious tests 
have been limited to a relatively small sample of cases. It 
is therefore necessary to judge the accuracy of the current 
approach and utilize this measure as a standard for evaluating 
the discriminant analysis methodology. The requirement, then, 
is an analysis of probabilistic forecasts of the 48 crises 
in our sample, using current EWAMS capabilities. 

Tables 4-3 and 4-4 supply the relevant information. The 
first set of figures summarizes the results of probabilistic 
forecasts for five indicators with an assumption of no lead/ 
lag; all 576 months within our data set are included and 
classified as "crisis” months if: 

• The probability is equal to or greater than 
.50 (for cooperation, conflict, and total 
activity Z-scores), or 

• Tension is 70.0 or greater, or 

• Hrel is .70 or greater 

Section 3.2.1 briefly discusses the "first-stage" 
extrapolative methodology; see also Daly and Davies (1978). 
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Table 4-3 


CLASSIFICATION RESULTS: EXTANT PROBABILITIES* 



HITS 

MISSES 

FALSE 

ALARMS 

CORRECT 

REJECTIONS 

SUCCESS 

PROBABILITY 

YULE’S 

Q 








TOTAL ACTIVITY 

Z-SCORE 

20 

28 

19 

509 

91.8 

.90 

COOPERATION 

Z-SCORE 

3 

45 

1 

527 

92.0 

.94 

CONFLICT 

Z-SCORE 

22 

26 

25 

503 

91.1 

.89 

TENSION 

28 

20 

73 

455 

83.9 

.79 

HREL 

5 

43 

2 

526 

92.2 

.94 


CLASSIFICATION SUCCESS FORECAST SUCCESS 

CRISIS NO CRISIS CRISIS NO CRISIS 

CASES CASES FORECASTS FORECASTS 


TOTAL ACTIVITY 


Z-SCORE 41.6 

COOPERATION 

Z-SCORE 6.3 

CONFLICT 

Z-SCORE 45.8 

TENSION 58.3 

HREL 10.4 


96.4 51.3 94.8 

99.8 75.0 92.1 

95.3 46.8 95.1 

86.2 27.7 96.2 

99.6 71.4 92.4 


‘Monthly; 10-year established Z-score tall; no lag; single-indicator probabilities (cooperation, 
total, and conflict Z-scores) or thresholds (75.0 tension, .70 Hrel). 
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'able 4-4 


LAGGED CLASSIFICATION RESULTS: EXTANT PROBABILITIES* 


FALSE CORRECT SUCCESS YULE’S 
HITS MISSES ALARMS REJECTIONS PROBABILITY Q 

TOTAL ACTIVITY 

Z-SCORE 7 41 12 468 90.0 .74 

COOPERATION 

Z-SCORE 0 48 1 479 90.7 -1.00 

CONFLICT 

Z-SCORE 10 38 15 465 90.0 .78 

TENSION 14 34 23 457 89.2 .78 


HREL 1 47 1 479 90.9 .82 


CLASSIFICATION SUCCESS FORECAST SUCCESS 


CRISIS NO CRISIS CRISIS NO CRISIS 

CASES CASES FORECASTS FORECASTS 

TOTAL ACTIVITY 

Z-SCORE 14.6 97.5 36.8 91.9 

COOPERATION 

Z-SCORE 0.0 99.8 0.0 90.0 

CONFLICT 

Z-SCORE 20.8 

TENSION 29.2 


96.9 40.0 92.4 

95.2 37.8 93.1 


'Monthly; 10-year established Z-score tail; 1-month lag; single- 
indicator probabilities (cooperation, total, and conflict Z-scores) 
or thresholds (75.0 tension, .70 Hrel). 
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The assumption is made that an analyst is advised to forecast 
a crisis if the probability for a given indicator is over 
one-half, because chances of a successful forecast are then 
over one-half. It is also assumed that an Hrel of .70 or a 
tension score of 70.0 is associated with a .50 probability of 
crisis as recommended by previous research (see, e.g., Daly 
and Bell, 1977). 

The results report the number of: "hits" (or crisis fore¬ 
casts which are correct); "misses" (or crises which were not 
classified as such); "false alarms" (or crisis forecasts which 
were incorrect)? and "correct rejections" (or the correct 
identification of a noncrisis month). Success Probability 
is the percentage of months correctly identified (as hits or 
correct rejections); Yule's Q is a similar measure of overall 
success with a maximum of 1.0. Classification Success refers 
to the percentage of crisis and noncrisis cases which were 
correctly classified: hits versus misses and correct rejec¬ 
tions versus false alarms are compared. Forecast Success 
reports the percentage of classifications which are correct: 
hits versus false alarms and correct rejections versus misses 
are compared. By far the most important statistics are: 

• Classification Success for crisis cases, which 
reports the likelihood that a crisis will be 
identified 


Note that a noncrisis month includes the precrisis 
months when no lag is introduced. 











• Forecast Success for crisis forecasts, which 
reports the likelihood that a crisis predic¬ 
tion is correct 

The former becomes high as misses are avoided; the latter 
increases as false alarms are avoided. These are more 
dynamic measures of success, are highly sensitive to 
success/failure rates, and assess in a summary fashion the 
capabilities of indicators as well as the inevitable tradeoff 
between misses and false alarms. 

The findings in Table 4-3 may be contrasted with a 
"rational decision model" forecasting method, which would 
have the following characteristics; 

• No hits or false alarms 

• Success Probability = 91.7 (no lag), 90.9 
(1-month lag) 

• 0.0% classification success for crisis cases 

The model would never forecast a crisis because the a 
priori probability of a crisis is very low (i.e., without 
information gleaned from the indicators, the likelihood of 
a crisis is one-twelfth in our data base—one of every twelve 
months is a crisis month—and less in the real world). Notice 
that the Success Probability is high; the "no crisis" fore¬ 
casts are almost always correct. 
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The performance of the current probabilities in the 
EWAMS is quite credible. The Total Activity Z-score correctly 
identifies over 40 percent of the crises and is more likely 
to "hit" than "false alarm." The Conflict Z-score is equally 
powerful, with a slightly better crisis classification success 
but a slightly lower crisis forecast success; the Conflict 
Z-score decreases misses at the expense of more false alarms. 
The Tension threshold performs credibly but is much more 
likely to false alarm and is inferior to the Conflict Z-score. 
The Cooperation Z-score and Hrel indicators miss most of the 
crises but seldom false alarm. The average indicator Success 
Probability is 90.2. 

Table 4-4 uses the precrisis I month as a lagged proba¬ 
bilistic forecast reference point. In lagged forecasts, we 
attempt to distinguish between the precrisis month and the 
(ten previous) noncrisis months. In effect, we wish to know 
if a month is, or is not, a precursor to the crisis month. 

The 48 crisis months within the data base are expunged for 
such analyses because they occur after the month of concern. 

In general, current EWAMS indicator probabilities perform 
poorly as lagged crisis warning tools. The number of "hits" 
is halved in most cases and the crisis classification success 
rate is greatly reduced. False alarms also become less fre¬ 
quent, but in only one case (tension) does the crisis forecast 
success increase, as does the overall Success Probability. 
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The average indicator reaches a crisis classification success 
of 32.7 with no lag and 13.3 with a one-month lag; average 
crisis forecast success similarly declines from 54.4 to 32.8. 
The average overall Success Probability for lagged forecasts 
is unchanged at 90.2, still slightly lower than a "rational 
decision model" which never offers a crisis forecast. 

These findings can serve as a standard for comparison 
with the results provided through applications of the discrim¬ 
inant analysis methodology. As noted earlier, the methodology 
seeks to replace a labor-intensive approach with a machine- 
based method of probabilistic forecasting and improve forecast 
accuracy or, at the very least, avoid losses in accuracy. The 
following section reports the results of the methodological 
tests. 


4.2.3 Discriminant analysis results . The most success¬ 
ful route to probabilistic crisis forecasting is through the 
use of coincident indicators which are themselves forecasted 
and which provide the information required for crisis warning. 
Coincident indicators are related to crises without a lead/lag 
relationship; they reflect the current situation and distin ¬ 
guish crises from noncrisis situations only contemporaneously 
In practice, coincident indicators can only be used retrospec¬ 
tively, i.e., after the data are collected, to identify a cri¬ 
sis; they must be forecasted in order to provide real-time, 
future-oriented warnings. The coincident indicator approach 
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therefore requires a sophisticated indication forecast method¬ 
ology as well as a crisis forecast methodology. However, if 
current indicators are incapable of providing advanced warning 
(are not leading indicators), this dual-step forecasting pro¬ 
cedure becomes necessary. The discriminant ana.lysis method¬ 
ology provides the second step: the linking of indicators to 
probabilistic crisis forecasts. 

Tables 4-5 and 4-6 report the results for tests which 
attempt to forecast crises within our data base using a 
coincident indicator approach. The results can be compared 
to Table 4-3, which also employs coincident indicators but 
reports results for a more limited set of probabilities (and 
thresholds). The tables include forecasts which are based on 
single-indicator probability estimates; in the discriminant 
analysis, each of the ten indicators is processed. 

Tables 4-5 utilizes a Bayesian prior probability of cri¬ 
sis equal to one-half; we assume complete lack of information 
in the absence of indicators and, hence, an inability to 
choose between a "crisis", or "no crisis" forecast. Table 4-6 
reports results when this prior is changed to 0.10 in order to 
represent or approximate the more conservative "rational" 
model (i.e., we assume that crises are very rare). In 
practice, the lower a priori will result in lower (posterior) 
probabilities, fewer false alarms, and more misses. A proba¬ 
bility of .4 is unusually high given an a priori probability 
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Table 4-5 


DISCRIMINANT CLASSIFICATION RESULTS: 

EQUAL PRIOR PROBABILITIES* 

FALSE CORRECT SUCCESS YULE’S 
HITS MISSES ALARMS REJECTIONS PROBABILITY' 0 

TOTAL ACTIVITY 23 25 43 485 88.2 .82 

TOTAL Z-SCORE 28 20 27 501 91.8 .93 

COOPERATION 16 32 43 485 87.0 .70 

COOP. Z-SCORE 22 26 36 492 89.2 .84 

CONFLICT 23 25 32 496 90.1 .87 

CON. Z-SCORE 21 27 21 506 91.7 .90 

TENSION 36 12 156 372 70.8 .75 

TEN. Z-SCORE 30 18 69 459 84.9 .84 

HREL 34 14 130 398 75.0 .76 

HREL Z-SCORE 25 23 56 472 86.3 .80 


CLASSIFICATION SUCCESS 8 FORECAST SUCCESS 3 

CRISIS NO CRISIS CRISIS NO CRISIS 

CASES CASES FORECASTS FORECASTS 

TOTAL ACTIVITY 47.9 91.9 34.8 95.1 

TOTAL Z-SCORE 58.3 94.9 50.9 96.2 

COOPERATION 33.3 91.9 27.1 93.8 

COOP. Z-SCORE 45.8 93.2 39.3 95.0 

CONFLICT 47.9 93.9 41.8 95.2 

CON. Z-SCORE 43 8 96.0 50.0 64.9 

TENSION 75.0 70.5 18.8 96.9 

TEN. Z-SCORE 62.5 86.9 30.3 96.0 

HREL 70.8 75.4 20.7 96.6 

HREL Z-SCORE 52.1 89.4 30.9 95.4 

* No lag; Z-Scores: 5-year established tail, by reference to crisis month. Single indicator 
results, monthly data. 

'Probability that a case or forecast is correctly identified (probability of a hit or correct 
rejection.) 

'Probability that a (crisis or no crisis) case will be classified correctly. 

'Probability that a (crisis or no crisis) forecast is correct. 
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Table 4-6 


DISCRIMINANT CLASSIFICATION RESULTS: 
10% PRIOR PROBABILITY OF CRISIS* 


FALSE CORRECT SUCCESS YULE’S 
HITS MISSES ALARMS REJECTIONS PROBABILITY 1 Q 


TOTAL ACTIVITY 12 36 9 519 

TOTAL Z-SCORE 12 36 9 519 


92.2 .90 

92.2 .90 


COOPERATION 4 44 6 

COOP. 2-SCORE 9 39 10 


522 

518 


91.3 .78 

91.5 .85 


CONFLICT 11 37 8 

CON. Z-SCORE 11 37 6 


520 

522 


92.2 .90 

92.5 .93 


TENSION** 0 

TEN. Z-SCORE 12 


48 0 

36 16 


528 

512 


91.7 — 

91.0 .83 


HREL 8 

HREL Z-SCORE 12 


40 17 

36 14 


511 90.1 .71 

514 91.3 .67 


CLASSIFICATION SUCCESS 2 FORECAST SUCCESS 3 


CRISIS NO CRISIS CRISIS NO CRISIS 

CASES CASES FORECASTS FORECASTS 


TOTAL ACTIVITY 25.0 98.3 

TOTAL Z-SCORE 25.0 98.3 


57.1 

57.1 


93.5 

93.5 


COOPERATION 8.3 

COOP. Z-SCORE 18.8 


98.9 40.0 92.2 

98.1 47.4 93.0 


CONFLICT 22.9 

CON. Z-SCORE 22.5 


98.5 57.9 93.4 

98.8 64.7 93.4 


TENSION 0.0 

TEN. Z-SCORE 25.0 


100.0 — 

97.0 42.9 


91.7 

93.4 


HREL 16.7 96.8 

HREL Z-SCORE 25.0 97.3 


32.0 

46.2 


92.7 

93.5 


*No lag; Z-Scores: 5-year established tail, by reference to crisis month. Single indicator 
results, monthly data. 

* ‘Tension results identical to always forecasting “no crisis," the rational decision given 
no indicator aid. Note; 91.7 success probability is a standard for comparison. 

'Probability that a case or forecast is correctly identified (probability of a hit or correct 
rejection). 

’Probability that a (crisis or no crisis) case will be classified correctly. 

’Probability that a (crisis or no crisis) forecast is correct. 
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of .1, yet a "no crisis" forecast is recommended; given an 
a priori of .5, the same posterior probability estimate is 
less normal and results in the same recommended forecast. 


The average indicator results are reported in Figures 
4-3 and 4-4. The average indicator success probability 
is 2 percent lower than the current probability success rate 
when the prior is .5 and 1.2 percent higher when the prior is 
.1; Yule's Q, which also measures overall success, is slightly 
lower in both cases. The Crisis Classification Success and 
Crisis Forecast Success statistics reported in Figure 4-3 are 
nearly identical to those in Table 4-3: approximately 30 
percent of the crises are "hit" and 54-55 percent of the 
forecasted crises are correct. When the Bayesian prior is 
reduced to .1, "false alarms" decrease and "misses" increase: 
nearly 50 percent of the forecasted crises occur, but 81 
percent of the crises are missed. While summary measures 
suggest a very slight decrease in the accuracy of discriminant 
probabilistic forecasts, the most important indicators suggest 
virtually no difference when the prior probability is .5; 
reducing the prior to .1 results in contradictory comparisons 
of the summary measures and a marked change from a "false 
alarm" propensity to a "miss" propensity. 

Reviewing the specific indicators, we may note that 
overall and specific success rates are generally greater for 
Z-scored versions of indicators; thus, the mixed findings when 
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Figure 4-3 


AVERAGE DISCRIMINANT CLASSIFICATION RESULTS: 
EQUAL PRIOR PROBABILITIES 
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Figure 4-4 

AVERAGE DISCRIMINANT CLASSIFICATION RESULTS: 
10% PRIOR PROBABILITIES 
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DATA 
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38.9 
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518.5 

98.2 
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we compare Table 4-3 to Figures 4-3 and 4-4 may be a function 
of including additional indicators. Less general comparisons 
yield the following results: 


• The total activity Z-score performs better 
overall (and better in most specific success 
rates) in the discriminant analysis results. 

• Crisis Classification Success for the 
cooperation Z-score improves in discriminant 
analysis, but Crisis Forecasting Success is 
reduced for an overall reduction in perfor¬ 
mance . 

• Overall, the conflict Z-score performance 
improves under the new methodology as Crisis 
Forecast Success increases greatly and Crisis 
Classification Success decreases slightly. 

• Tension generally performs poorly overall in 
discriminant analysis results and tends to 
false alarm a great deal while seldom missing 
a crisis when the prior is .50. 

• Hrel or uncertainty performs poorly in these 
results, as Crisis Classification Success 
increases and Crisis Forecast Success decreases. 


Indicators which are not associated with probabilistic 
forecasting capabilities are also capable of generating 
crisis probabilities. These include: 


• Total activity , which performs credibly in spite 
of a false alarm tendency (with a prior of .5). 

• Cooperative activity , which has a better overall 
Success Probability than Hrel, but performs 
more weakly than most indicators. 

• Conflict activity , which "hits" more crises than 
its Z-score version, but offers more false 
alarms. 
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• Tension 2-score , which is far superior to raw 
tension by offering fewer false alarms. 

• Hrel Z-score , which generally outperforms its 
unstandardized version. 

These indicators, in general, are capable of generating 
probabilistic crisis forecasts which are as accurate as the 
previous set of indicators. 

The results suggest that the EWAMS can be provided with 
a probabilistic crisis forecasting method (discriminant 
functions) which can utilize any single indicator to gener¬ 
ate probabilities and the utility of these probability esti¬ 
mates will remain at moderate-to-high levels. Moreover, the 
accuracy of forecasts based upon these probabilities remains 
strong at either of the two levels of a priori probabilities 
(.5 and .1).* ** The use of the methodology can replace the 
current approach to probability estimation without loss of 
validity and with gains in efficiency and the user may be 
informed regarding the trustworthiness of any probability 
in retrospect. These conclusions apply to coincident 

• 

This is relatively meaningful if we propose to allow 
analyst/user input as a determinant of the Bayesian prior. 

** 

For example, the analyst may be told of the "proba¬ 
bility of a false alarm" and the "probability of a miss" 
for each indicator when the .50 probability cutoff is used 
retrospectively. 


101 



FT 

i 



indicator probabilities of crisis, where current procedures 
are most accurate. 


*Results for the lagged tests are not reported here; 
see Rossa (1979c) for this phase of the research. 


102 






5.0 COMPARATIVE SOURCE PERFORMANCE 


The work of Charles McClelland (1968, 1972), who 
developed the World Event Interaction Survey (WEIS) under 
DARPA/CTD sponsorship in the 1960s, provides a key basis 
for the Early Warning and Monitoring System (EWAMS). WEIS 
is an events data coding system which characterizes event/ 
interactions or events in terms of an actor-initiator (who), 
target-recipient (whom), subject (what), and time (when). In 
the analytic EWAMS, the user may retrieve WEIS data from NYT 
and/or MAG, examine crisis probabilities, obtain tabular, 
graphic, and textual output, and conduct keyword searches. 

The events data movement in comparative and international 

politics has spawned a number of source comparison studies. 

** 

The major examples will be briefly surveyed here. 

In a frequently cited comparative international events 
analysis, Azar et al. (1972) examined the New York Times 
Index and a prominent regional source (the Middle East 
Journal ) with respect to their coverage of Egyptian-Israeli 
relations from 1955-1958; they concluded that the two sources 

* 

This section is based on Hopple (1980a, 1980c). 

** 

See especially Burgess and Lawton (1972); general 
findings and specific studies are reviewed in Peterson 
(1975) and Scolnick (1974). 
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differed significantly in the frequency of events reported. 

The regional source reported more cooperative events, sug¬ 
gesting that global sources may provide a distorted portrait 
of reality by overemphasizing the dramatic and the conflictual. 

Other regional-global source comparisons have been 
reported in the literature on international politics (e.g., 
Burrowes, 1974). In addition, extensive inquiry has been 
undertaken to assess the relative performance of regional and 
global sources in the area of intranational conflict analysis. 
In a comparison across eleven Caribbean countries (1948-1964), 
Doran et al. (1973) compared Deadline Data (a global events 

chronology) and the Hispanic American Report ; significant 

* 

source differences were uncovered. 

in a more detailed study, Jackman and Boyd (1979) analyzed 
two Africa-specific sources ( Africa Diary and African Recorder ) 
and three global sources ( New York Times Index , Keesing's 
Contemporary Archives , and Facts on File ). They discovered 
that the structure of internal violence (based on factor 
analytic patterns) varied across sources. However, the sub¬ 
stantive results for hypotheses concerning the determinants 
of collective protest and coups did not differ significantly.* ** 

* 

See also Hazelwood and West's (1974) comparison of 
internal conflict data for twenty Latin American nations 
(1955-1960). 

** 

On the coup model, see also Jackman (1978). 
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Jackman and Boyd (1979: 453) offered the unexpected con¬ 

clusion that "...the importance of using multiple sources 
and the distinction between global and regional sources is 
easily overstated." 

The Jackman and Boyd (1979) study thus implies that the 
effects of source differences on research on internal African 
politics are not noteworthy. Their focus, however, was basic 
research on the determinants of internal instability. From 
the perspective of intelligence analysis, where there is more 
concern with short-term (i.e., daily or weekly) and specific 
phenomena, the outlook is less favorable. 

For example, Cholawsky et al. (1979) compared four sources 
with respect to their daily coverage of the Rhodesia-Zimbabwe 
area: 

• Foreign Broadcast Information Service (FBIS), 

Series for Sub-Saharan Africa , Volume VIII 

• New York Times (NYT) 

• Times of London (TOL) 

• Christian Science Monitor (CSM) 

The time frame was the period May-July, 1978. 

Generally, Cholawsky et al. (1979) identified no sig¬ 
nificant differences among the four sources regarding the 





proportions of violent and nonviolent stories. However, the 
CSM reported the fewest items and the "sample" in that source 
was seriously skewed. On the volume criterion, FBIS and 
TOL contained the largest numbers of stories. When violent 
incidents were broken down by target group, there were dis¬ 
tinct variations; FBIS favored Rhodesian troops and black 
civilians and TOL devoted more attention to white civilians. 
NYT and TOL featured more balance between groups than FBIS or 
CSM. FBIS, TOL, and NYT appeared to be "adequate" or "good" 
sources; CSM was clearly "inadequate." 

Aside from regional-global source comparisons, the 
empirical work has focused on "equivalent" sources. For 
example. Hill and Fenn (1974) compared NYT and the Times of 
London (TOL) as sources of (international) WEIS data for the 
three year period from 690101 to 711231. 

Overall, Hill and Fenn (1974) discovered that NYT and 
TOL were generally consistent with respect to conflict and 
crisis sequences. Secondly, the newspapers exhibited dif¬ 
ferent reporting styles during non-conflict periods. 

Thirdly, NYT tended to report more events than TOL. 

Regarding the first conclusion, Hill and Fenn (1974) 
presented findings for four case studies; 

• India-Pakistan (1971) 
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• Jordan-PLO (1970) 


• The Middle East 

• Vietnam 

While NYT was more thorough in its coverage (especially of 
the Middle East and Vietnam), the two sources displayed 
analogous crisis reporting styles. For example, the dis¬ 
tinct peaks and valleys of the Indo-Pakistani conflict showed 
up clearly and comparably. 

Semmel (1978) profiled four U.S. elite dailies (NYT, 
Miami Herald , Chicago Tribune , Los Angeles Times ) in terms 
of the amounts and types of foreign news items featured in 
each. Random samples (25 week-day editions) were drawn from 
the last three months of 1974 and 1977. The four newspapers 
displayed similar (although not identical) patterns. 

Perhaps the most ambitious study is Gerbner and Marvanyi 
(1977), who examined the foreign news coverage of the daily 
press of different social systems. Sixty dailies in nine 
countries (representing the West, East, and Third World) 
were systematically compared for the week of May 24, 1970. 
Both elite and popular newspapers (as well as mass-circula¬ 
tion and small types) were represented for the U.S., the 
Soviet Union, and the seven other nations. The focus was 
the volume and proportion of news about the external world 
in each country and newspaper. 
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The allocation of attention to different parts of the 
world varied sharply by newspaper and social system; the 
latter differences were especially marked. In effect, the 
five press systems (US, Soviet Union, Western Europe, Eastern 
Europe, Third World) all presented unique portraits of the 
world. This generalization applied to both the sheer volume 
of news coverage and the distribution of attention to 15 
world "regions." The U.S. newspapers highlighted Western 
Europe, Asia and the Far East, North America, and the Middle 
East; these four regions accounted for two-thirds of the 
U.S. perceptual map. 

5.1 Rationale 


No single newspaper or press system mirrors reality in a 
direct or simple fashion; Gerbner and Marvanyi (1977) 
emphasized quite correctly that all news systems exhibit 
selectivity and purpose. 

However, as was pointed out in an early comparative 
assessment of the NYT and MAG EWAMS data sources (Hopple, 
1979c: 5), a tradeoff between the goals of cost and validity 

must be made. The use of system-specific sources for every 
country in the world would yield a rich WEIS data set, but 
the time and cost involved preclude such an option. The 
reliance on a single data source, on the other hand, is a very 
risky strategy. No single source—even one as prestigious 
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as NYT—can be expected to provide a "foolproof" snapshot 
of all facets of international reality; biases and omissions 
are inevitable. 


The problem of comparing sources is compounded by the 
fact that there is no external, infallible standard for 
determining what constitutes the "real world." If we knew, 
for example, that Africa accounts for 7 percent of the measur¬ 
able international events occurring during a given period while 
Western Europe initiates 12 percent of the events and receives 

14 percent of the total, we could compare sources against this 
** 

common, "objective" standard. That, unfortunately, is pre¬ 
cisely the kind of information which is unavailable. 


Some event data researchers have concluded that NYT 
is the best single source; researchers who face severe con¬ 
straints of time or money may therefore be justified in 
relying solely on NYT. However, this argument is signifi¬ 
cantly more convincing from the perspective of basic 
researchers concerned with retrospective analyses of aggregate 
trends and patterns than it is from the vantage point of 
intelligence analysts whose focus is specific contexts and a 
current or short-term future time frame. 

★ it 

In her interesting analysis of newsgathering and 
filtering by Times of London "gatekeepers" (i.e., home office 
newsmen and foreign staff correspondents and stringers), 
Peterson (1979) created her own universe of (hypothetical) 
events and elicited judgments about them in a mail question¬ 
naire. As she noted, comparing sources "does not answer the 
question as to what has been systematically excluded from all 
sources [Peterson, 1979: 121]." 

*** 

Even if it were available, it would not negate the fact 
that the transitory and longer-term interests of different 
countries shape attention patterns (e.g., Britain's emphasis 
on the EEC, the U.S. awareness of Iran in 1979, etc.). To 
take several extreme examples, a crisis between Mongolia and 
China would be of less concern to the U.S. intelligence 
community than one in southern Africa, which in turn would 
not be ranked as high as a crisis between Greece and Turkey. 
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Thus, there is no omniscient, unbiased observer against 
whom both NYT and MAG can be compared. Furthermore, the 
envisioned role of EWAMS in the intelligence analysis or 
I&W (Indications and Warning) process should be explicitly 
recognized. The Early Warning and Monitoring System is 
designed to: 


• Lengthen the warning lead time before 
serious crises of direct or indirect 
relevance to U.S. security interests 
erupt 

• Systematize the information retrieval 
and monitoring dimensions of the I&W 
process 

• More generally, enhance and supplement 
(but not replace) the judgment and in¬ 
tuition of the intelligence analyst 


The third generalization is the most pertinent in the 
context of this discussion. The system is driven by the 
objective of aiding the human analyst in the performance of 
his or her tasks of monitoring and assessing incoming streams 
of data (the multitude of simple and complex indications) and 
making estimates and forecasts. 


In a very fundamental sense, an I&W analyst is an (ap¬ 


plied) empirical scientist testing hypotheses. It is a canon 
of scientific research that, with the soft data so typical of 
political and many other forms of social scientific analysis, 







a multimethod or multi-indicator strategy is more likely to 

* 

produce reliable, valid results. 

Jackman and Boyd's (1979) Africa data source comparison 
advances the argument that the impact of source variations on 
substantive findings (i.e., whether or not different sources 
yield different results in tests of hypotheses oor models) has 
been neglected in prior research; the focus in much of the 
extant work has been on simple volume comparisons. Their own 
research suggests that source variations do not seem to be 
associated with nonequivalent substantive results—at least 
with respect to the determinants of mass protest behavior 
and coups in sub-Saharan Africa. 

Similar logic can be applied to the assessment of the 
utility of event data-based indicators in intelligence 
analysis. Simple volume comparisons do not necessarily war¬ 
rant the inference that MAG and NYT are either equivalent or 
noncomparable sources. 

The critical litmus tests, in fact, are the monitoring 
and warning indicator comparisons chronicled in the latter 
part of this section. The total activity, tension, and 
uncertainty indicators are employed here to "evaluate" the 

See Campbell and Fiske (1959) and Hopple (1980b: 
ch. 2). 
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two sources and generate some preliminary evidence about the 
viability of the EWAMS dual source structure. 


Viability in this context has connotations different 
from those implied by classical scientific research. The 
latter revolves around the assessment of a model in general 
terms (e.g., how much of the total variance is explained? 

How statistically significant are the correlation co¬ 
efficients or betas?, etc.). Discrepancies are often down¬ 
graded or ignored; deviant cases are treated as statistical 
outliers which should be removed or which might be explained 
more adequately in another iteration or by a more sophisti¬ 
cated model. 


From the perspective of an intelligence analyst, however, 
each case is at least potentially important and incongruities 
become critical. When nothing is happening (i.e., neither 
source indicates unusual or noteworthy activity) and this can 
also be verified by other data streams, the analyst is 
justified in a no-crisis forecast (i.e., a correct rejection). 
When both (and perhaps other) sources converge on a warning, a 
crisis forecast is more credible (i.e., the system "hits"). 


Probabilities, Z-scores, and other EWAMS indicators 
(e.g., CAP, Special Purpose Indicators, etc.) could have been 
added to the design but would have increased the complexity 
of the analysis considerably. As the discussion below should 
establish, the data and indicators surveyed here should be 
sufficient for the purpose of assessing the general utility 
of a dual (or multiple) source approach. 
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The other two states of affairs—misses and false alarms— 


are of course much more problematic. 

A dual- or multi-source system quite clearly exhibits an 
increased potential for false alarming (as does a multi¬ 
indicator, single source system). Generally, however, false 
alarms are preferred over misses. 


The reasoning which underlies this preference can be 
illuminated by drawing on an example from research on psycho¬ 
physiology and biocybernetics. In the extended quotation 
below from Donchin (1979), brackets are employed to insert 
material which relates to the intelligence analysis and cri¬ 
sis warning processes. 


Consider an operator lor I&W analyst] presented with 
a series of items which he must classify as belonging 
to one of two categories Ie.g., crisis or not-crisis]. 
The large marjority of the items [or discrete events] 
with which he is presented are not important Ii.e., 
constitute the ongoing and normal stream of events]. 
However, randomly interspersed within this series of 
neutral items are important items which must be prop¬ 
erly classified Ii.e., monitoring and warning situa¬ 
tions]. One can imagine a number of such examples, 
ranging from the detection of defects in prescription 
pills on a production line through the identification 
of enemy aircraft from amongst other signals on a radar 
screen. In the example I shall describe, the operators 
were presented on the cathoderay tube screen of a com¬ 
puter terminal with a series of names, 80% of which were 
female names, 20% male names. Their task was to press 
a button with one hand on the appearance of a male name, 
and with the other hand on the appearance of a female 
name. 


It occurred to us that it may be possible to 
incorporate a biocybernetic error-correction algorithm 





in a system that would allow subjects to respond as 
fast as they wished while allowing the computer to 
correct for errors using an algorithm based on P300. . . 
Whenever "thought" preceded action (as it presumably 
does whenever P300 latency is shorter than reaction 
time), the subject tended to be correct. This inter¬ 
pretation of the data suggests that it should be 
possible to identify the trials in which the subject 
erred by evaluating the difference between reaction 
time and the P300 latency. McCarthy, Jutas, and 
Donchin (in preparation) developed such an error- 
correcting biocybernetic algorithm and applied it to 
the data obtained from the ten subjects that were 
assigned the task of discriminating between male and 
female names. Briefly, this decision-aiding procedure 
operated as follows. Whenever the subject gave an 
"important" response, the item was accepted as 
"important." If the subjects make a "neutral" 
response, the EEG was examined for the presence or 
absence of the P300. If P300 was absent, the trial 
was accepted as neutral. Those trials on which the 
subjects responded "frequent," but which were identi¬ 
fied as containing a P300 by the first tier of the 
analysis, were subjected to further analysis in which 
the latency of the P300 was compared to the reaction 
time obtained on that trial. If reaction time was 
longer than P300 latency, the trial was accepted as 
neutral. That is, the subject's response was accepted 
as valid. However, if reaction time was shorter than 
P300 latency, the trial was considered to be a candidate 
for a misclassified "important" item. A third analysis 
was then conducted which utilized information about 
differences in waveform between correctly and incor¬ 
rectly classified rare trials. Following this third 
tier of the analysis, some trials were considered to 
be definitely "important" and the rest were accepted as 
neutral. 

The efficacy of this procedure can be evaluated 
as follows. Its benefits consist of all the important 
trials that were classified by the subject's response 
as neutral but were reclassified by the biocybernetic 
algorithm as important. Those are, in fact, the errors 
corrected by the biocybernetic algorithm ["misses," 
in EWAMS terminology]. The cost consists of neutral 
trials which were classified correctly by the subject, 
but which the algorithm now considers important. The 
results obtained for the ten subjects show a very sub¬ 
stantial benefit. In fact, most of the errors com¬ 
mitted by the subjects were retrieved by the algorithm, 
albeit at the cost of increased errors in classifying 
the frequent items. Of course, in any such procedure, 
the costs and benefits must be evaluated in relation 
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to the specific task. The absolute number of mis - 
classified "frequents" following the algorithm is very 
large. The absolute number of retrieved "important” 
items is relatively small . If, however, it is the case, 
and we believe it often is, that in such tasks the 
operator's main responsibility is to avoid misclassify- 
ing the important items, then the cost-benefit analysis 
performed suggests that a substantial improvement in 
the system's performance is gained by the incorporation 
of the biocybernetic error-correcting code. [Emphasis 
added] 


Similarly, in intelligence analysis, the central goal 
is to "avoid misclassifying the important items." This means 
that misses, when a crisis is misclassified as a routine 
situation or noncrisis, should be avoided despite the fact 
that such a goal will be accompanied by an increase in the 
number of false alarms (analogues to neutral trials incor¬ 
rectly considered to be important). 

In the I&W process, then, an identical cost-benefit 
configuration is desirfible. Crisis forecasting is one of 
those situations in which the object of concern is very rare 
(i.e., the absolute number of retrieved "important" items 
can be expected to be relatively small in any case); clearly, 
however, the "operator's" main responsibility is the avoidance 
of misses. 


5.2 Analytic I&W 


This section presents empirical data from the MAG and NYT 


sources for the first six months of 1979. Earlier source 








I 

comparisons (e.g.. Hopple, 1979b, 1979c) were limited to the 
three months of October, November, and December, 1978. In 
addition to extending the time frame, this analysis also avoids 
the potentially confounding impact of the switch from the 
Times of London or TOL (used in October and November of 1978) 
to MAG (the source for December, 1978). Thus, the NYT-MAG 
comparison for the last three months of 1978 actually 
involved NYT and TOL for the first two months and NYT and 
MAG for the third. This January-June, 1979 assessment 
compares NYT and MAG for all six months. 

5.2.1 Aggregate results . Overall totals for the two 
newspapers are presented in Table 5-1. For the six-month 
period from 790101 to 790630, NYT contained 2243 discrete 
event/interactions and the MAG total was 2249. On a monthly 
basis, the deceptive nature of these very similar grand 
totals is clearly revealed. 

For example, NYT yielded 356 events in January of 1979 
and MAG produced only 266. During the next month, the 
comparable Ns were 492 and 409. NYT was slightly ahead of 
MAG in April (296 to 273) and May (339 to 320). MAG totals 
exceeded those for NYT in two months: March (516 for MAG 
and 457 for NYT) and June (465 and 303 for MAG and NYT, 
respectively). 

The TOL labor strike began in December of 1978. 
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Table 5-1 


TOTAL VOLUME: JANUARY-JUNE, 1979 a 

TOTAL COOP CON 



NYT 

MAG 

NYT 

MAG 

NYT 

MAG 

790101 

356 

266 

227 

180 

129 

86 

790201 

492 

409 

305 

223 

187 

186 

790301 

457 

516 

284 

254 

173 

262 

790401 

296 

273 

165 

101 

131 

172 

790501 

339 

320 

231 

206 

108 

114 

790601 

303 

465 

190 

307 

113 

158 

Total 

2243 

2249 

1402 

1271 

841 

978 


a Coop=cooperation; con=conflict; NYT- New York Times ; 
MAG=Manchester Guardian. 


As the standard WEIS event list in Table 5-2 shows, the 
dichotomies of cooperation and conflict can be broken down 
into 22 discrete event-interaction categories; 10 are coopera¬ 
tive in nature and 12 conflictual. The 22 categories range 
from yield to force. A variety of verbal and physical forms 
of behavior constitutes the 22 general and 63 specific WEIS 
coding categories. 


Source differences emerge immediately when the NYT and 
MAG event frequency breakdowns are displayed (see Table 5-3). 
Highlighted in both data sets are the categories comment, con¬ 
sult, agree, accuse, and force. However, it is clear that the 
actual event-interactions are far from identical. At the 
extreme ends of the "continuum," both yield and force acts 
should be fairly straightforward and easily codable; their 
levels of reliability should consequently be higher than the 


117 




Table 5-2 


PT 



1 


THE WEIS EVENTS CODING SCHEME 


1. YIELD 

011 Surrender, yield to order, submit to 
arrest, etc. 

012 Yield position, retreat, avacuate 
013 Admit wrongdoing, retract statement 

2. COMMENT 

021 Explicit decline to comment 
022 Comment on situation-pessimistic 
023 Comment on situation-neutral 
024 Comment on situation-optimistic 
025 Explain policy or future position 

3. CONSULT 

031 Meet with, at a neutral site, or send note 

032 Visit, goto 

033 Receive visit, host 

4. APPROVE 

041 Praise, hail, applaud, condolences 
042 Endorse other policy or position, give 
verbal support 

5. PROMISE 

051 Promise own policy support 
052 Promise material support 
053 Promise other future support 
054 Assure, reassure 

6. GRANT 

061 Express regret, apologize 
062 Give state invitation 
063 Grant asylum 

064 Grant privilege, diplomatic recognition. 

de facto relations, ate. 

065 Suspend negative sanctions, truce 
066 Release endfor return person or 
property 

7. REWARD 

071 Extend economic aid 
072 Extend military aasistanee 
073 Give other assistance 

8. AGREE 

081 Moke substantive agreement 
082 Agree to future action or procedure, 
agree to meet, to negotiate 

9. REQUEST 

081 Ask for Information 
092 Ask for policy assistance 
093 Ask for material aseistance 
094 Request action, call for 
09$ Entreat, plead, appeal to 

10. PROPOSE 

101 Offer proposal 

102 Urge or suggest action or policy 


11. REJECT 

111 Turn down proposal, reject protest 
demand, threat, etc. 

112 Refuse, oppose, refuse to allow 

12. ACCUSE 

121 Charge, criticize, blame, disapprove 

122 Denounce, denigrate, abuse 

13. PROTEST 

131 Make complaint (not formal) 

132 Format complaint or protest 

14. DENY 

141 Deny an accusation 

142 Deny an attributed policy, action, 
role or position 

15. DEMAND 

150 Issue order or command, insist, demand 
compliance, etc. 

16. WARN 

160 Give warning 

17. THREATEN 

171 Threat without specific negative 
sanctions 

172 Threat with specific negative sanctions 

173 Threat with force specified 

174 Ultimatum, threat with time limit and 
negative sanctions specified 

18. DEMONSTRATE 

181 Nonmliitary demonstration, walk-out on 

182 Armed force mobilization, exercise 
andfor display 

19. REDUCE RELATIONSHIP 

191 Cancel or postpone event 

192 Reduce routine Internetional activity, 
recall officials, etc. 

193 Reduce economic eid or militery 
assistance 

194 Halt negotiations 

195 Break diplomatic relations 

20. EXPEL 

201 Order personnel out of country 

202 Expel organization or group 

21. SEIZE 

211 Seize position or possessions 

212 Detain or arrest person<s) 

22. FORCE 

221 Noninjury destructive ect 

222 Nonmilitary injuryfdestruction 

223 Military engagement 
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NYT-MAG EVENT FREQUENCIES: JANUARY-JUNE, (1979) 
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other categories. It is therefore instructive to compare NYT 
and MAG with respect to simple monthly frequencies of yields 
and acts of force: 


Yield 


Force 


Date 

NYT 

MAG 

NYT 

MAG 

790101 

1 

0 

31 

22 

790201 

0 

2 

40 

32 

790301 

4 

8 

37 

25 

790401 

2 

0 

26 

34 

790501 

3 

2 

21 

24 

790601 

0 

2 

23 

28 


Assuming that the yield and force categories are very 
reliable (at least compared to more ambiguous types of 
international behavior—i.e., if a yield or force event is 
reported, the data collector will have little or no problem 
identifying and coding it), then the discrepancies displayed 
above suggest that NYT and MAG contain a significant amount 
of nonoverlapping material. Both sources show that yields 
are rare; however, MAG reports none in January while NYT 
contains 1, NYT has none for February while MAG reports 2, 
and so forth. The totals are different for all six months. 
The same holds for force, with NYT exceeding MAG during the 
first three months and the Guardian showing more acts of 
force during April, May, and June. 

The lack of equivalence recurs throughout Table 5-3. 
Especially noteworthy is the difference between comment 
totals, with NYT consistently yielding more comments than 


120 




MAG (e.g., 114 versus 38 in January, 123 versus 42 in 
February, etc.). Other salient variations include: 

• Consult (June, 1979: 59 for NYT and 117 

for MAG) 

• Agree (June, 1979: 22 for NYT and 73 

for MAG) 

• Accuse (March, 1979: 63 for NYT and 105 

for MAG) 

• Demonstrate (March, 1979: 3 for NYT and 

19 for MAG) 

The significance and implications of these variations 
can be easily exaggerated and misinterpreted. For example, 
the lack of identity at this very global (world-world) level 
of scanning may conceal a considerable amount of similarity 
at lower levels of analysis. The comparison of overall totals 
in this fashion does not permit a direct assessment or even a 
reasonable indirect estimate of the actual amount of overlap 
in the two data sets. 

Furthermore, it is entirely possible that MAG and NYT may 
perform similarly at the country pair level and especially 
in situations of high tension and severe conflict and crisis. 
This was the general conclusion, it should be emphasized, in 
Hill and Fenn's (1974) three-year comparison of NYT and TOL 
WEIS data. The focus will therefore shift to a less global 
level of analysis in the following section. 
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5.2.2 Country-to-world results . Analyses of country-to 
world interaction profiles are presented in this section. 
Marshall Pulley of the EWAMS computer science staff developed 
special standalone software which was used on the Demonstra¬ 
tion and Development Facility's PDP 11/70 minicomputer to pro 
duce event frequencies for each country in the world for the 
period 790101-790630.* Both cooperative and conflictual 
activity levels and events initiated and received were com¬ 
bined; the results, presented in Table 5-4, therefore consist 
of all events sent from and received by every country and 
other WEIS actor during the first half of 1979. 

Thus, NYT reported a total of 824 events for the U.S. 
(initiator and recipient)? the corresponding MAG total was 
527. Table 5-4 reveals that NYT tends to report more events 
for certain actors and MAG yields larger totals for others. 

Of 146 actors, the MAG volume is greater in 90 in¬ 
stances; the NYT total is larger than (or equal to) MAG in 
56 cases. Especially noteworthy differences can be isolated 
systematically by listing those countries for which NYT 
exceeded MAG by 5 or more events and vice versa. 

The Demonstration and Development Facility is DARPA/ 
CTD's computer facility, where all EWAMS analyses are per¬ 
formed on the on-site 11/70. 
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Table 5-4 


EVENTS SENT AND RECEIVED BY COUNTRY: 
JANUARY-JUNE (1979) 


ewhts neouDcr 

Coaparative mi ConfUctual 

mai - rma 


Initiator mi feclpJmt 


UN 




RKtplant 


aalU 

albwla 

tfugoalawta 


Iceland 

nato 

•ae 

Inti orj (w, etc) 
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Using this criterion, NYT’s volume was "significantly" 
higher for 24 countries. These include the USA itself, the 


USSR, Iran, the Koreas, the Chinas, Vietnam, India, and 
Pakistan. When the Middle EAstern actors on the list (UAR, 
Lebanon, Israel, Yemen, South Yemen, the PLO, and the Arab 
League) are taken into account, it is clear the NYT tends to 
favor the three superpowers as well as prominent dyads from 
the Cold War era (China and Taiwan, North and South Korea, 

India and Pakistan) and the Middle East. The Camp David 
period of late 1978 and its protracted aftermath undoubt¬ 
edly account at least in part for the salience of Egypt and 
Israel. 

Whereas the NYT total is greater than MAG's 24 times, 
the MAG source exceeds NYT's total by 5 or more events in 
the case of 50 actors. Quite clearly, MAG's attention is 
more dispersed and less focused on "key actors." Generally, 
the analyst interested in smaller countries would be advised 
to consult the Guardian . 

Aside from the United Kingdom itself, the Guardian pays 
more attention to former British colonies (with the significant 
exceptions of India and Pakistan). Among these are Canada, 
Jamaica, and Grenada in the Western Hemisphere, a number of 
former African colonies (Uganda, Kenya, Zambia, Rhodesia, 

South Africa), and Singapore. 
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In addition, MAG features Western Europe much more than 
NYT does (including Austria, Italy, Denmark, the EEC, the 
Netherlands, France, Spain, and even West Germany). Surpris¬ 
ingly, West Berlin also makes this list, as do several Eastern 
European nations (Czechoslovakia, Yugoslavia, Bulgaria, and 
Rumania). In contrast, only one European country (aside from 
the Soviet Union) appears on the NYT list: Switzerland. 

MAG also devotes significantly more attention to Africa 
and other parts of the Third World. In addition to the ex¬ 
colonies in sub-Saharan Africa, MAG reports significantly 
greater numbers of events for both former Portuguese colonies 
(Angola and Mozambique) as well as for Zaire, Ethiopia, the 
Zimbabwe Liberation Front, and Botswana. 

In the Middle East, the following countries are repre¬ 
sented on the MAG list: Morocco; Libya; Iraq; Syria; Jordan; 
and Kuwait. Also, three subnational actors in the volatile 
Middle East receive more emphasis in MAG than NYT: the Kurds 
and the Lebanese Christians and Moslems. 

Afghanistan, Cambodia, Indonesia, and even the U.S. ally 
Thailand and the former U.S. colony the Philippines join 
Singapore as countries which elicit more coverage from the 

Until early 1980, the Zimbabwe code was used for the 
anti-Rhodesia liberation forces; it is now used as the code 
for the country and the Rhodesia code is no longer employed. 
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Guardian than from NYT. Finally, Japan's MAG activity level 
surpasses the corresponding NYT total. 

The greater U.S. emphasis on its superpower colleagues 

and the generally more concentrated nature of NYT's coverage 

* 

are both confirmed by other research. Even Latin America is 
relatively neglected by NYT; only Mexico and the fairly re¬ 
cent hotspot Nicaragua are represented among the 24 actors 
for which the Times exceeds the Guardian by 5 or more events. 

5.2.3 Country pair results . NYT versus MAG compari¬ 
sons for 9 selected country pairs are featured in Tables 5-5 
(activity) and 5-6 (tension and uncertainty). These dyads 
include ones which are almost invariably prominent in world 
affairs (US-USSR; US-China; China-USSR; Israel-UAR) as well as 
recent crisis pairs (Iran-USA; Uganda-Tanzania? China-Vietnam) 

and two centers of ongoing hostility and "permanent crisis" 

** 

(Rhodesia-Zimbabwe and Israel-PIO). 

As Table 5-5, which profiles total activity, cooperation, 
and conflict patterns, shows, there are only 12 out of 54 
volume comparisons which go above the "significance" criterion 
of 10. The most noteworthy of these include the U.S. and China 

See, e.g., Semmel's (1978) previously mentioned compari¬ 
son of U.S. elite newspapers. 

** 

The "permanent crisis" designation, of course, applies 
to Rhodesia-Zimbabwe from the 1960s up until recently. 


127 







Table 5-5 


SELECTED COUNTRY PAIR ACTIVITY LEVELS: 
JANUARY-JUNE, 197S a 





Table 5-6 
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i 

in January (30 events from NYT and only 2 from MAG), Iran and 
the U.S. in February (26/NYT and 8/MAG), and China-Vietnam 
(16/NYT and 3/MAG in January; 44/NYT and 26/MAG in February). 

This pattern clearly implies that source differences may 
be minimized at the level of active and/or crisis-prone coun¬ 
try pairs, a conclusion that is buttressed by Hill and Fenn's 
(1974) NYT-TOL analysis. However, the results for both ten¬ 
sion and H-rel (Table 10) are somewhat less encouraging. 

For tension, 23 of the 54 scores vary by more than 10 
points. Especially deserving of attention are Rhodesia- 
Zimbabwe and Iran-U.S.; for both pairs, MAG tension scores 
tend to be appreciably higher than NYT scores (particularly 
for the former pair in January, April, and June and the latter 
in the last four months of the period). For Iran-USA, in 
fact, MAG forecasts a crisis in May and depicts a relation¬ 
ship of consistently higher tension throughout the period. 

Instances when only MAG or NYT data cross the monitor¬ 
ing or warning tension threshold can be illuminated. There 
are 12 pairs which conform to this pattern: 


Month 

Pair 

NYT 

MAG 

790101 

chn-usr 

80 

0 

790201 

" 

69 

81 

790401 

" 

80 

50 

790101 

rho-zim 

22 

69 

790401 

" 

69 

92 

790601 

" 

50 

88 



I 


790301 

irn-usa 

0 

61 

790401 

" 

25 

67 

790501 

" 

56 

80 

790301 

uga-taz 

36 

66 

790301 

chn-vtn 

57 

73 

790401 

isr-plo 

67 

84 


In 8 of these 12 instances, one source forecasts a crisis 
(i.e., surpasses the threshold of 70) while the other does not. 
The most striking examples include China and the Soviet Union 
in January and Iran and the U.S. in May. 


Of the 54 H-rel or uncertainty comparisons, there are 25 
differences greater than the absolute value of .10. Many of 
these are technically but perhaps not "meaningfully" signifi¬ 
cant. However, note the .28 and .53 (February) and .36 and 
.60 (June) for Israel and the UAR. The Iran-U.S. scores for 
March are also distinctive (.16/NYT and .63/MAG); both tension 

and H-rel generated monitoring advice from MAG but not NYT 
** 

during the month of March. 


The tension and uncertainty score configurations both 
illustrate the utility of a dual source warning and monitoring 


* 

The EWAMS indicators have been used primarily for track¬ 
ing and forecasting severe crises (i.e., episodes of hostility 
and conflict which may escalate into serious violence or war). 
However, there is some evidence that the uncertainty indicator 
is also useful for profiling the dynamics of "hard bargaining" 
between former enemies (i.e., situations of protracted enmity 
which are evolving into detente relationships). 

** 

This is one of the few situations in which both core 
indicators from source A signal a "watch" (i.e., monitoring) 
or crisis warning while neither indicator from source B does. 




system. In the typical basic research analytical endeavor, 
these very specific differences would be of less concern; in 
the context of a system which has been designed for transfer 
to a "user community" of intelligence analysts, the case- 
specific discrepancies which have surfaced assume much more 
significance. 

5.3 Automated I&W 


A series of real-time tests and analyses of the Early 
Warning and Monitoring System (EWAMS) is currently in progress. 
Central to this task is the question; how does the EWAMS 
perform in a current, real-time mode? Using EWAMS data, 
indicators, and probabilities for projection (one week into 
the future) and short-term retrospective analysis (one week 
into the past), how accurately does the system monitor and 
warn? In order to conduct these tests, the EWAMS data col¬ 
lection operation at IPPRC is now "real-time." New York Times 
or NYT data are one day behind on any given date; Manchester 
Guardian or MAG data show a lag of five to seven days. 

The purpose of this section is to compare in a systematic 
fashion the retrospective performance of NYT and MAG in terms 
of EWAMS monitoring and warning of international crises. 

5.3.1 Hotspots lists . The rank order correlation 
statistic rho is used to evaluate the conflict actor rankings 
generated by MAG and NYT during each month of 1979. Rho 
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varies from a perfect negative correlation (-1.00) to a per¬ 
fect positive one (+1.00). If the two rankings are identical, 
rho = +1.00; if the rankings are reversed, rho = -1.00. The 
formula for computing rho is: 

r rank = 1-— S - P - 

N (N 2 -l) 

where D = differences for the two ranks; and 
N = number of total items ranks. 

The example in Table 5-7 illustrates the procedure. 

The rhos are presented in Table 5-8 for each of the 12 
months in 1979, for the entire year, for the first 6 months 
of 1979, and for the second 6 months of the year. The 
following rationale was employed to assess the rhos. 

First of all, the least desirable pattern would be one 
in which the correlation was highly negative, indicating the 
existence of an inverse relationship. If, in other words, 

NYT ranks an actor high, MAG tends to rank it low (or not at 
all). A rho of +1.00, in contrast, would imply that MAG is 
redundant and therefore superfluous. The third type of 
result is when the rhos are fairly strong but not perfect 
correlations; rhos above .50 but below .80 or .90 are "pre¬ 
ferred." This range indicates that the two sources are not 
completely discrepant but, simultaneously, some advantage 
does accrue by tracking both sources. 
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Table 5-7 


THE CALCULATION OF RHO: AN ILLUSTRATION 



NYTRANK 

MAG RANK 

DIFFERENCE (D) 

Sc P 

U.S.A. 

1 

1 

- 

- 

IRAN 

2 

2 

— 

— 

U.S.S.R. 

3 

3 

— 

— 

U.K. 

4 

4 

— 

— 

U.N. 

5 

12.5 

7.5 

56.25 

ZIMBABWE 

6 

5 

1 

1 

NATO 

7 

12.5 

5.5 

30.25 

LIBYA 

8 

12.5 

4.5 

12.25 

PLO 

9 

12.5 

3.5 

12.25 

ISRAEL 

10 

8 

2 

4 

SOUTH AFRICA 

12.5 

6 

6.5 

42.25 

RHODESIA 

12.5 

7 

5.5 

30.25 

VIETNAM 

12.5 

9 

3.5 

12.25 

EGYPT 

12.5 

10 

2.5 

6.25 

SD 2 = 215 





N 14 





rho =1 - 6 x 215 = 
14( 14 2 - 1) 

1 1290 = 1 

2730 

1 - .47 = .53 








Table 5-8 



NYT VS. MAG: RANK ORDER CORRELATIONS 
FOR CONFLICT HOTSPOTS LIST 


Time Frame RHO 


01/79 

.61 

02/79 

.65 

03/79 

.57 

04/79 

.36 

05/79 

.52 

06/79 

.47 

07/79 

.43 

08/79 

.52 

09/79 

.55 

10/79 

.23 

11/79 

.66 

12/79 

.53 

01/79-06/79 

.82 

07/79-12/79 

.74 

01/79-12/79 

.79 


Generally, the rankings conform to the third pattern. 
The overall (12-month) rho is .79 and the rhos for the first 
and second six month periods are .82 and .74, respectively. 
At this high level of aggregation, the two source lists tend 
to be similar in nature. 


NYT and MAG converge in identifying the U.S. as the most 
conflictual actor for all 12 months and for the two half-year 
periods. Generally, the actors within the top five are the 
same, although sometimes there are slight differences within 
the top five lists (e.g., Israel is #2 on the NYT list and #3 
on the MAG list for the first half of 1979). The top five 
rankings are identical in 7 of 15 instances and vary within 
the top five in 7 of 15 cases; in only one case, then, is an 
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actor in the top five list from one source but not the other. 
This is the PLO, which is ranked fifth on the NYT list for the 
second half of 1979 but ranked ninth by MAG. 

The rankings from sixth to tenth display similar patterns, 
with the bulk of the rankings being identical or close. The 
only consistent tendency is MAG's relative emphasis on Great 
Britain, which makes only the MAG list twice (Janu^ry-June, 

1979 and January-December, 1979) and is ranked sixth by NYT and 
third by MAG during the July-December time span. The monthly 
results show more variation, with the rhos ranging from .23 
(October) to .66 (November). Eight of the 12 monthly rhos are 
above the arbitrary .50 cutoff and all are below the (equally 
arbitrary) .80 ceiling, suggesting that inter-source similarity 
is discernible but not overwhelming. 

During the early part of 1979, the China-Vietnam crisis 
was a dominant issue on the agenda of international affairs; 
this was clearly reflected in both the MAG and NYT lists. 
Especially in February, when both ranked China first and 
Vietnam third, the Sino-Vietnamese crisis was captured by 
the two elite public media sources. This crisis and its pro¬ 
tracted aftermath remained "hot" through the month of July. 

Both sources also tended to identify the tense Uganda- 
Tanzania relationship—an ongoing conflict arena which had 
flared up in the fall of 1978 and culminated with the ouster 
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of Idi Amin. After April, the Uganda-Tanzania conflagration 
subsided. 


The third salient hotspot pair during 1979 was Israel and 
Egypt. The Camp David breakthrough of late 1978 led to an in¬ 
tense period of bargaining in the ensuing months. In addition 
to the superpowers, Rhode siaT~7!iirtbabwer~and- th e PL O — also—fre._ . 
quently appeared on the lists. Toward the latter part of the 

year, Iran surfaced; it was ranked third on both lists for 
* 

the month of August. 

According to the rhos, the most discrepant months were 
April and October. During the former month, the U.S., Israel, 
and Egypt were ranked in the top three by both NYT and MAG. 
However, four NYT countries did not appear on the MAG hotspots 
list: Tanzania (ranked fifth); China (seventh); Uganda 

(eighth); and Lebanon (ninth). Rhodesia, Pakistan, Zimbabwe, 
and the European Economic Community were ranked by MAG but 
not by NYT. 

October featured only four actors which appeared on both 
source lists; the U.S. (ranked 1 by NYT and 2 by MAG); 

Zimbabwe (ranked 2 by NYT and 3 by MAG); Britain (ranked 3 by 
NYT and 1 by MAG); and the Soviet Union (ranked 4 by NYT and 

* 

Iran first showed up in March 1979, ranking sixth on the 
MAG list; it appeared again on the MAG list in May (with a 
rank of seventh) and was on the NYT list (at tenth) and the 
Mag list (at sixth) in June. 
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6 by MAG). While NYT highlighted Egypt, Israel, Rhodesia, 
and Iran, MAG focused on Vietnam, Thailand, France, China, 
the EEC, and the UN. 

November was characterized by a much higher level of 
agreement, with the seizure of the U.S. embassy in Tehran 

-early'in the month detonating the Iran-U.S. crisis. NYT 

ranked Iran first and the U.S. second; MAG agreed on the 
identity of the actors but reversed the order. The upsurge 
in activity in the Rhodesia-Zimbabwe (Patriotic Front)-United 
Kingdom arena was reflected in both sources. The November 
rho of .66, the highest of the year, was approximated only 
by the rhos of January and February, when the Sino-Vietnamese 
crisis dominated world affairs. 

5.3.2 Monitoring and warning country pair lists . The 
country pairs which comprise the monthly monitoring and 
(crisis) warning lists are presented in Table 5-9. The 
entries represent the number of indicators which crossed the 
thresholds for monitoring (of a total of five possible) and 
for warning (of a total of three possible). Since the 
monitoring search goes back three months from the alert date, 
for any given month the maximum total would be 15 thresholds; 
the maximum for the more stringent warning search, which in¬ 
volves only the immediate preceding month, is three thresh¬ 
olds. 
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Table 5-9 


MONITORING AND WARNING COUNTRY PAIR LISTS 
SOURCE COMPARISON (Cont'd.) 


C. 790301 MAG 

M W 


NTT 


M W 


D. 790401 


M W M W 


1. usa-usr 

2. usa-chn 

3. usa-unk 

4. usa-isr 

5. usa-uar 

6. usa-gmw 

7. usa-frn 

8. usr-chn 

9. usr-unk 

10. chn-vtn 

11. unk-uar 

12. unk-vtn 

13. unk-rho 

14. isr-uar 

15. usa-uno 

16. usa-cht 

17. usa-irn 

18. chn-cht 

19. isr-vtn 

20. isr-uno 

21. isr-irn 

22. usr-irn 

23. vtn-unp 

24. vtn-irn 

25. uar-irn 

26. uno-irn 


3 0 2 

3 0 4 

10 0 

5 11 

4 0 1 

2 0 0 

2 0 0 

4 12 

2 0 0 

6 18 

10 0 

10 0 

10 0 

2 0 0 

0 0 1 

0 0 4 

0 0 5 

0 0 6 

0 0 1 

0 0 2 

0 0 3 

0 0 1 

0 0 2 

0 0 2 

0 0 2 

0 0 2 


0 1. 

0 2 . 

0 3. 

0 4. 

0 5. 

0 6 . 

0 7. 

1 8. 

0 9. 

1 10 . 

0 11. 

0 12 . 

0 13. 

0 14. 

0 15. 

0 16. 

0 17. 

0 18. 

0 19. 

1 20. 

0 21 . 

0 22 . 

0 23. 

0 24. 

0 25. 

0 26. 

27. 

28. 
29. 


usa-usr 1 
usa-chn 3 
usa-isr 4 
usa-unk 1 
usa-uar 3 
usa-irn 4 
usa-plo 5 
usr-chn 3 
usr-unk 2 
usr-uar 1 
usr-irn 1 
chn-vtn 6 
isr-uar 2 
isr-irn 1 
isr-plo 3 
unk-uar 1 
unk-vtn 1 
uar-irn 1 
chn-uno 0 
usr-vtn 0 
isr-vtn 0 
isr-uno 0 
vtn-uno 0 
vtn-irn 0 
uar-plo 0 
uar-mlg 0 
uno-irn 0 
irn-plo 0 
irn-mlg 0 


0 2 0 

0 4 0 

0 2 0 

0 0 0 

0 2 0 

0 5 0 

0 0 0 

13 1 

0 0 0 

0 0 0 

110 
2 8 0 

0 10 

0 4 0 

0 3 0 

0 0 0 

0 0 0 

0 4 0 

0 10 

0 10 

0 10 

0 2 0 

0 3 0 

0 2 0 

0 10 

0 10 

0 2 0 

0 3 0 

0 4 0 
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E. 790501 

1. usa-usr 

2. usa-isr 

3. usa-uar 

4. usa-chn 

5. usa-irn 

6. usa-plo 

7. usr-uar 

8. usr-chn 

9. usr-irn 

10. isr-uar 

11. isr-lrn 

12. isr-plo 

13. uar-unk 

14. uar-irn 

15. chn-vtn 

16. vtn-unk 

17. isr-uno 

18. chn-uno 

19. vtn-uno 

20. uar-plo 

21. uar-mlg 

22. uno-irn 

23. plo-irn 

24. irn-mlg 


MONITORING AND WARNING COUNTRY PAIR LISTS: 
SOURCE COMPARISON (Cont'd.) 

MAG NYT F. 790601 MAG 

M W M W MW 

2030 1. usa-isr 2 0 

40 20 2. usa-uar 1 0 

30 20 3. usa-usr 1 0 


3 0 10 

4 0 4 0 

5 0 10 

10 0 0 

4 0 3 1 

10 10 
3 0 10 

10 4 0 

3 14 0 

10 0 0 

2 0 2 0 

6 0 9 0 

10 0 0 

0 0 10 

0 0 3 0 

0 0 2 0 

0 0 10 

0 0 2 0 

0 0 2 0 

0 0 3 0 

0 0 5 0 


4. usa-chn 1 0 

5. usa-plo 5 0 

6. usa-irn 4 1 

7. isr-uar 4 0 

8. isr-chn 1 0 

9. isr-unk 1 0 

10. isr-plo 4 1 

11. isr-uno 2 0 

12. uar-usr 1 0 

13. uar-plo 2 0 

14. uar-irn 2 0 

15. usr-chn 3 0 

16. usr-irn 1 0 

17. chn-vtn 6 0 

18. unk-uno 1 0 

19. usa-uno 0 0 

20. isr-leb 0 0 

21. uar-mlg 0 0 

22. chn-uno 0 0 

23. uno-vtn 0 0 

24. uno-mlg 0 0 

25. uno-leb 0 0 

26. plo-leb 0 0 
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Table 5-9 


MONITORING AND WARNING COUNTRY PAIR LISTS: 
SOURCE COMPARISON (Cont'd.) 


G. 790701 MAG 

M W M 

1. usa-usr 203 

2. usa-chn 100 

3. usa-vtn 100 

4. isr-unk 100 

5. isr-uar 401 

6. isr-usr 100 

7. isr-chn 100 

8. isr-plo 314 

9. isr-uno 201 

10. unk-usr 100 

11. unk-vtn 300 

12. unk-frn 100 

13. unk-uno 100 

14. uar-usr 100 

15. uar-chn 102 

16. uar-plo 300 

17. uar-uno 10 0 

18. usr-chn 201 

19. usr-uno 10 0 

20. chn-vtn 304 

21. chn-uno 102 

22. frn-uno 200 

23. usa-isr 002 

24. usa-plo 001 

25. isr-vtn 001 

26. isr-leb 004 

27. uar-mlg 002 

28. uno-mlg 001 

29. uno-leb 004 

30. plo-leb 001 

31. mlg-vtn 001 


H. 790801 MAG 

W MW 

0 1. usa-isr 1 0 

0 2. usa-uar 1 0 

0 3. usa-usr 1 0 

0 4. usa-chn 1 0 

0 5. usa-vtn 2 0 

0 6. usa-jap 1 0 

0 7. unk-isr 1 0 

1 8. unk-usr 1 0 

0 9. unk-vtn 3 0 

0 10. unk-frn 1 0 

0 11. unk-jap 2 0 

0 12. unk-rho 2 0 

0 13. isr-uar 3 0 

0 14. isr-usr 1 0 

0 15. isr-chn 1 0 

0 16. uar-usr 1 0 

0 17. uar-chn 1 0 

0 18. uar-frn 1 0 

0 19. usr-chn 3 0 

0 20. chn-vtn 3 0 

0 21. frn-jap 1 0 

0 22. usa-mlg 0 0 

0 23. usa-nic 0 0 

0 24. isr-uno 0 0 

0 25. isr-plo 0 0 

0 26. isr-vtn 0 0 

0 27. uar-mlg 0 0 

0 28. uar-plo 0 0 

0 29. uno-mlg 0 0 

0 30. vtn-mlg 0 0 

0 31. mlg-nic 0 0 
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NYT 

M W 

1 0 

1 0 

2 0 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

1 0 

0 0 

0 0 

0 0 

2 0 

0 0 

1 0 

3 1 

0 0 

1 0 

7 0 

2 0 

3 1 

1 0 

1 0 

1 0 

1 0 

3 0 

1 0 
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19 



MONITORING AND WARNING COUNTRY PAIR LISTS: 
SOURCE COMPARISON (Cont'd.) 
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:able 5-9 


MONITORING AND WARNING COUNTRY PAIR LISTS: 
SOURCE COMPARISON (Cont'd.) 


K. 791101 MAG NYT 

M W M W 

1. usa-unk 3000 

2. usa-isr 1020 

3. usa-usr 3030 

4. usa-irn 1020 

5. usa-vtn 1000 

6. unk-rho 2020 

7. unk-zim 8151 

8. unk-vtn 2000 

9. unk-frn 2000 

10. usr-irn 1000 

11. rho-zim 4041 

12. vtn-frn 1000 

13. usa-chn 0010 

14. usa-mlg 0010 

15. isr-uar 0010 

16. isr-mlg 0010 

17. irn-mlg 0040 

18. chn-mlg 0010 

19. mlg-rho 0030 


L. 791201 MAG NYT 

M W M 

1. usa-irn 5 36 

2. usa-usr 3 04 

3. usa-vtn 1 00 

4. unk-irn 3 03 

5. unk-zim 10 0 5 

6. unk-rho 203 

7. unk-vtn 1 00 

8. unk-frn 2 00 

9. irn-usr 201 

10. irn-frn 200 

11. zim-rho 404 

12. usr-chn 1 00 

13. chn-vtn 2 10 

14. chn-frn 1 00 

15. usa-isr 001 

16. irn-uar 001 

17. irn-plo 003 

18. irn-chn 001 

19. isr-uar 001 

20. isr-plo 002 
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Several general comments should be made about the find¬ 
ings displayed in Table 5-9. First, the EWAMS default cri¬ 
teria were in effect for each monthly run. This means that 
conflict hotspots lists were generated for each alert date; 
based on the top ten conflict actors, the system examined 
every possible combination of country pairs for both the 
monitoring and warning lists. It is possible that two 
countries could have been in a serious crisis situation but 

would not have appeared on the list because either or both 

★ 

failed to make the initial input (hotspots) list. 

Secondly, the nature of the monitoring lists is somewhat 
different from the warning list. While the latter is a crisis 
projection list, the monitoring pairs represent a "watch 
advice" signal. Included are tense situations and potential 
crises? also registered on many lists are situations of 
unusual cooperation between formal or potential allies, 
generally in response to a perceived threat from a third 
party. 

It should be noted in this context that the monitoring 
thresholds include total activity probability (of .5 or 
higher), cooperation probability (of .5 or higher), and Hrel 
or uncertainly (.5 or higher). For example, Israel and the 
UAR appeared on the MAG January 1979 list, with three of the 

* 

The user may add or delete actors to or from the 
initial list. 
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monitoring thresholds exceeded. The preceding month, of 
course, featured the Camp David breakthrough. All three 
indicators were Hrel, which varies from 0 to 1.0, with higher 
scores indicating greater uncertainty or higher complexity; 
the Israel-UAR Hrels were .55 (October 1978), .58 (November 
1978), and .62 (December 1978). 

Another illustrative example is the February 1979 MAG 
monitoring list. Both the U.S.-France and the U.S.-West 
Germany pairs appeared; for the former, the total probability 
was .63 and the cooperation probability was .87 (both in 
January 1979) while for the latter, the total probability was 
.50 and the cooperation probability was .76 (also both in 
January 1979). 

Aside from the obvious caveat that alert list pairs, 
thresholds, and indicators should be analyzed in context 
rather than mechanically or arbitrarily, these results 
suggest that EWAMS may be of some value as a mechanism for 
tracking alliance phenomena.* Although the system has been 
designed as a crisis and conflict monitoring and warning 
system, in some cases the probabilities and indicators may 
reveal significant patterns pertaining to cooperative pheno¬ 
mena. In certain contexts, for example, Hrel may be indica¬ 
tive of a cooperative breakthrough; periods of impending 

* 

For an application to the Middle East, see Daly and 
Andriole (1979). 
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detente are probably as complex as relations which are 
escalating to a crisis. 

A recent case provides a very graphic illustration. 

After a history of tension and conflict, Rhodesia and 
Zimbabwe (the Patriotic Front) achieved a cooperative break¬ 
through and elections were held in early 1980. In September 
of 1979, the cooperation probability was .95 while the con¬ 
flict and total probabilities were only .24 and .40, respec¬ 
tively. In January and February of 1980, cooperation again 
experienced a probabilistic upsurge—to .92 and .81, re¬ 
spectively. Meanwhile, the January and February total proba¬ 
bilities were .77 and .54; the comparable conflict probabil¬ 
ities were .55 and .44. 

As Table 5-9 suggests, 1979 was an unusually crisis- 
plagued year. Including the three superpower dyads (because 
of their intrinsic importance to world affairs and also be¬ 
cause of the tense Sino-Soviet relationship and the dete¬ 
riorating U.S.-Soviet relationship), both the MAG and NYT 

data as well as intuitive judgment suggest that there were 

** 

13 distinct crisis or potential crisis arenas. These in¬ 
clude: 

The EWAMS probabilities, for example, are based on 48 
crisis dyads from 1966-1978, suggesting an average of 3 or 4 
crisis country pairs (rather than crises per se) a year. 

** 

This includes the U.S.-Israel-UAR arena. 
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China-Vietnam 


• Israel and the PLO 

• Zimbabwe 
-U.K. -Zimbabwe 
-U.K.-Rhodesia 

-Rhode sia-Zimbabwe 

• U.S.-Iran 

• Iran-others 

• U.S.-U.S.S.R. 

• U.S.S.R.-China 

• U.S.-China 

• U.S.-China/Taiwan 

• China-China/Taiwan 

• U.S.-Nicaragua 

• Camp David and its aftermath 
-U.S.-Israel 

-U.S.-Egypt 
-Egypt-Israel 

• U.S.-U.K, 


The last pair warrants a comment. The U.S. and Britain 
are listed only because of a small number of events in August 
of 1979. For the alert date 790901 for the MAG source, there 
were three monitoring thresholds crossed (tension = 56; 

Hrel = .5; conflict probability = .87) and one warning thresh¬ 
old crossed (conflict probability = .87). 
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The dispute revolved around Northern Ireland. Early in 
August, the British Foreign Office expressed increasing appre¬ 
hension about the growing influence of the Irish American 
lobby in the U.S., specifically with reference to an alleged 
extra arms sale to the Royal Ulster Constabulary. The U.S. 
State Department denied that there had been a policy change. 
Things heated up when New York Governor Hugh Carey invited 
British and Irish representatives to attend talks in the U.S. 
on the future of Northern Ireland, an overture which the 
British government rejected. 

A comparison of MAG and NYT with respect to the monthly 
lists reveals several arenas commonly identified as well as 
some which one or the other source tended to focus on. In 
a^._ition, there were numerous idiosyncratic deviations, with 
both sources selecting out particular pairs not identified by 
the other. 

Consensus was highest on the very salient crisis pairs 
of 1979: the U.S.-Iran; China-Vietnam; U.K.-Rhodesia-Zimbabwe; 
Israel-PLO. The two sources were very similar in their read¬ 
ings of the U.S.-Iran situation in April, May, and December. 

MAG crossed four monitoring thresholds in April and May while 
NYT exceeded five thresholds in April and four in May. In 
December, MAG generated five monitoring threshold readings 
and three for warning; NYT featured six monitoring and three 
warning readings. 
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However, NYT tended to identify the Iranian-U.S. arena as 


t 


conflictual when MAG was silent. On three occasions—May, 
September, and October—NYT recorded three or more monitoring 
readings, whereas MAG had one in October and none in the other 
two months. In contrast, MAG featured four monitoring read¬ 
ings and one for warning in January of 1979; NYT was silent. 

NYT also was more prone to illuminate tense or unusual 
relations between Iran and other countries. Several examples 
are displayed below. They all show the higher (and not 
surprising) level of attention on the part of the U.S. 
prestige newspaper. However, both sources did pick up the 
U.K.-Iran pair in December. 


Alert Date 

Monitoring 

Warning 


MAG 

NYT 

MAG 


790401 

3 

3 

0 


790501 

3 

4 

1 

0 

790601 

4 

5 

1 

1 

790701 

3 

4 

1 

1 


In addition to the actual crisis of 1979, the super¬ 
powers were tracked—and in generally congruent ways—by 
both MAG and NYT. From May through the end of the year, both 
sources reported several monitoring threshold crossings for 
the U.S.-Soviet pair. The Sino-Soviet tension of the first 
half of 1979 registered in both sources? from January to May, 








both reported several monitoring threshold crossings a month. 
In addition, NYT featured one crisis warning for 790201; both 
recorded one such warning for 790301 and again for 790401; 

NYT alone yielded one crisis warning for 790501. 

Several very serious situations from the U.S. perspective 
were captured by NYT but not by MAG. Especially salient in 
this category were the U.S.-Taiwan and Taiwan-China pairs 
in February and March, after the U.S. recognition of Peking. 

In all four cases, NYT reported four, five, or six monitoring 
readings while MAG contained none at all. The other case 
reflected only in NYT was the U.S.-Nicaragua pair during 
August of 1979, with seven monitoring readings in NYT and 
none in MAG. 

Overall, MAG tended to reflect more of the U.S.-Israeli- 
Egyptian dynamics. In January, for example, it reported five 
watch advice readings (compared to one in NYT) for U.S.-Israel; 
in February, MAG generated four such readings (again, compared 
to one in NYT). 

One other idiosyncratic feature of the MAG profile 
should be mentioned; not surprisingly, the British source 
yielded readings for the U.K. much more frequently than did 
NYT. Of 41 occasions, Britain was on both lists only six 
times and was on NYT and not MAG only once. 
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Crude summary statistics reveal the overall patterns 
for MAG vis-a-vis NYT. Of 309 discrete readings (i.e., all 
country pairs which crossed at least one monitoring threshold 
during one of the month time frames), 227 were situations in 
which MAG was 0 and NYT was 1 or more (or vice versa). 

There were 58 readings for which both reported at least one 
threshold crossed but the frequencies differed (e.g., NYT 
with 2 thresholds surpassed and MAG with 1); 20 of the 309 
involved comparisons for which the difference across sources 
was four or more. Finally, 23 pairs were identical in terms 
of number of indicators (assuming a minimum of one).** 

The warning readings logically fall into four types: 
both sources zero; MAG and NYT both above zero but with 
unequal numbers of thresholds exceeded; MAG and NYT both 
above zero and equal in magnitude; one source is zero and 
the other is one or more. Of 309 readings, 276 were cases 
which conformed to the first criterion (i.e., neither 
source yielded one or more crisis warnings). There were no 
cases when both broke the threshold but with different 
numbers of indicators. On seven occasions, both surpassed 


This includes pairs for which one source was 0 and 
the other 4 or more. 

** 

This of course does not mean that the indicator 
values—or even the indicators per se—were identical; the 
comparison here assumes that what is crucial is the crossing 
of a threshold. Identical scores of 3, for example, could 
involve different indicators, values, and even months. 
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the threshold(s) with equal number(s) of indicators. 

Finally, 26 times one source reported at least one warning 
whereas the other reported none. 

These findings are not troublesome in the sense that 
crises are very rare and crisis warnings should therefore be 
rare. Although indicator readings were equivalent in fre¬ 
quency only seven times and source A reported one or warn¬ 
ings and source B reported none 26 times, these results are 
neither inexplicable nor disturbing per se. Most of the 
latter involved instances when source A reported that one 
threshold had been exceeded and source B failed to note any 
crossings; this accounts for 23 of the 26 cases. These can 
safely be regarded as "false alarms" in the overwhelming 
majority of instances. 

5.4 Conclusions 


The results presented in this section should be regarded 
as tentative in nature. Work is currently being conducted by 
IPPRC on FBIS (Foreign Broadcast Information Service) data 
and by Mathtech on classified (cable traffic) data; both 


* 

The three exceptions included: two MAG thresholds 
for China-Vietnam and none for NYT (790401); two NYT 
thresholds for Iran-Multilateral Group and none for MAG 
(790901); two NYT thresholds for U.K.-Iran and none for 
MAG (791201). The second was concerned primarily with 
Iranian charges against and warnings to Western foreign 
news reporters. 
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efforts involve data collection only for sub-Saharan 
African international affairs. Included are key African dyads 
(e.g., Angola-South Africa, Ethiopia-Somalia, etc.) as well 
as events sent by the superpowers to relevant African 
countries. 

The FBIS and classified data patterns will be compared 
systematically with those generated by MAG and NYT. Very 
preliminary findings suggest that the former two sources 
yield a significantly greater volume than either of the 
latter two; the classified data are especially "data-rich." 
Comparative source performance profiles, however, cannot be 
delineated until more research is conducted. 
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6.0 FUTURE RESEARCH 


The Early Warning and Monitoring System (EWAMS) has 
progressed from a prototype to a fully developed system. At 
the same time, modifications and enhancements are constantly 
being made. These include both "cosmetic" and other user- 
oriented refinements (e.g., help routines) as well as 
research-based additions. An example of the latter is 
threat networks, discussed in the next section. 

6.1 Threat Networks 


The EWAMS has traditionally been dyadic in focus. In 
other words, the emphasis has been on pairs of actors 
(country pairs, two JCS regions, a region and a country); 
this applies to both analytic I&W and automated I&W. The 
crisis warnings generated by the latter, for example, consist 
of country pairs. The Country Activity Profile (CAP) fea¬ 
ture of EWAMS deals with country-level indicators (see 
Hopple and Snyder, 1979). 

A comprehensive I&W warning and monitoring system should 
include indicators at several distinct levels of analysis: 

• Country performance and activity measures 

• Dyadic indicators 
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• Networks 


• Systemic scans 

Networks refer to clusters of countries, situations 
when conflict and tension "spread" and spill over. This is 
a common syndrome in international politics. The India- 
Pakistan crises and war of 1971, for example, involved not 
only the central protaganists; the United States, Soviet 
Union, and China were peripheral but Clearly involved key 
actors as well. 

Preliminary research has already been conducted on the 
subject of threat networks? the key proposition is that, 

after a threat becomes present in the relations between two 

* 

antagonists, a specific subsystem will often develop. 

Threat subsystems embrace the central actors and the ever- 
increasing web of allies and adversaries. 

The Iranian and Afghan crises of late 1979 and early 

** 

1980 are ideal candidates for threat network analysis. The 
Iran-U.S. embassy crisis eventually "spread," involving a 
number of U.S. allies and others. The Afghan crisis soon 
became a U.S.-Soviet affair, eliciting attention from China, 
Pakistan, Iran, and various Western European and Middle 

*See Rothe (1980b) for details. 

Details on these cases are provided in Rothe (1980a). 
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Eastern countries. It could be argued that the Iranian and 
Afghan crises became "fused threat networks," with nations 
reacting to the "arc of crisis" in the Middle East and 
South Asia. 

For illustrative purposes, seven actors (the United 
States, Soviet Union, Iran, Afghanistan, Pakistan, China, 
and India) can be examined for the weeks of 791021, 791028, 
and 791104 (Figure 6-1). These weeks are displayed here in 
three directed graphs or digraphs. Each digraph is labelled 
by a D (digraph) and the beginning date of the week. 

Most striking is that the first digraph (D 791021) has 
absolutely no relations. Although each of these nations may 
be interacting with a variety of other nations, none of them 
is interacting with each other. No threat network exists at 
this particular point in time. In the second digraph 
(D 791028), actions take place within three country pairs 

(CHN-►USR, AFG-►USA, IRN- - * USA) , but the actions are 

in one direction. While this appears to be the beginning of 
a threat network, the relations among the dyads are not 
reciprocal. The subsystem finally appears in the last digraph 
(D 791104). During this time period, the Iranian students 
seized the American embassy and tense relations between the 
two nations began. 
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Figure 6-1 

ILLUSTRATIVE THREAT NETWORKS 
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The threat network analysis is pursued through the week 
ending on January 27, 1980 (800127) in Rothe's (1980a) 
detailed study, which develops and tests some quantitative 
indicators for threat network analysis and compares the 
1979-1980 network results with the findings for the 1971 
India-Pakistan network. Research is continuing in this 
area; we expect some more definitive results by early 1981. 

6.2 Other Research 


Daily and weekly data (raw scores and deviation scores 
or Z-scores) have been available to the EWAMS user for some 
time. Our current emphasis is on generating conflict 
probabilities for daily and weekly data. 

The current monthly probabilities provide the foundation 
for this aspect of our research program. We are thus employ¬ 
ing the discriminant function analysis methodology. However, 
problems unique to daily and weekly data (e.g., the frequent 
absence of data) haver-die feated ^ j ~ methodological enhance¬ 

ments. 

Given the focus and tasks of the I&W analyst, the concern 
is clearly with the short-term future—the next several days 
or weeks. A 30-60 day time horizon is generally the maximum 
interest of the typical prospective user of EWAMS. Therefore, 
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the ability to generate daily and weekly probabilities re¬ 
mains paramount on our research agenda. 

Missing from the extant EWAMS indicator base is the 
global level. A systemic scan indicator would permit the 
analyst to search for crisis probabilities from the vantage 
points offered by distinct levels of analysis. A systemic 
scan indicator would monitor all global political activities, 
thus providing a summary measure for the current state of 
affairs for the world. 

The rationale for systemic (and other levels) of indica¬ 
tors flows from the nature and structuring of the typical 
I&w analyst's tasks. Typically, responsibility is diffused 
on the basis of country and area foci? this division of labor 
is characteristic of DIA, CIA, State, and other elements of 
the national security community concerned with monitoring, 
forecasting, and I&W. While this organizational framework 
makes sense, given the voluminous amount of available informa¬ 
tion and the need to become immersed in the context and back¬ 
ground of a country, .country pair, and/or regional arena, it 
also poses the danger of myopia and actor- or region-specific 
biases. A common syndrome, for example, is for an analyst to 
"assume" the comparative visibility and "permanent" crisis¬ 
like nature of his or her region. Systemic (and comparative 
regional) indicators could provide a mechanism for correcting 
or at least minimizing such biases. 
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The envisioned indicator "staircase" would feature 
country-level (e.g., CAP and/or variations of it), country 
pair or dyadic, regional, and systemic levels. Ordinarily, an 
analyst could "descend" from the systemic. Is the "world" 
unusually active or tense? If so, from where is the trouble 
emanating? Comparatively, how tension-saturated are regions A 
and B or dyads A and B within a region? 

Encouraging preliminary analysis has already been com¬ 
pleted (see Rothe, 1979). Two currently available systemic 
scan possibilities were systematically compared and evaluated: 
the Event Flow Indicator (EFI) and the Pattern Recognition 
Algorithm (PRA). Among the conclusions were: 

• Both systemic indicators (EFI and PRA) monitor 
well 

• PRA is more rigorous than EFI 

• PRA is easier to interpret and therefore more 
user-oriented 

• PRA is capable of being used with extant fore¬ 
casting methods 

• PRA is slightly better for warning because of 
its sequence capacity 

An advantage of the PRA scan is that it is adaptive. 

Each new week of data that is classified is added to the past 
weeks in the classification scheme. The statistical para¬ 
meters of each class are always changing as a result of the 
addition of new patterns. If there is a gradual change in 
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the structure of world politics, this will be reflected in 
the changing parameters of each class in the typology. If 
there is a sudden change in the ordering of international 
classes generating new patterns of data which are not sim¬ 
ilar to any of the original class, a new class can be added 
to the typology. 

It is in this sense that the PRA scan is adaptive 
to changes in the dynamics of the international system; 
each week, the new patterns that are added to the classifi¬ 
cation scheme slightly change the parameters of the patterns. 
Similarly, when a new pattern of data is classified, it is 
going to change the matrix of transition probabilities. 

Thus, the probabilities projecting where the system will go 
next also adapt and hopefully reflect the change character¬ 
istics of new orderings in the international system. 

During FY 81, the utility of developing and incorporating 
into EWAMS a systemic scan indicator will be explored. Retro¬ 
spective and real-time tests will be conducted to determine 
the feasibility and desirability of integrating such an 
indicator. 

Testing of the early warning indicators with real-time 
and daily data remains one of our highest priorities. Ten 
years of WEIS data (1966-75) from the New York Times are 
now fully supplemented by daily updated data. 








Since the WEIS data are available and are now integrated 
into the system on a daily basis, a real-time analysis of our 
indicators must continue. There is no substitute for tests 
and evaluations of indicators on a real-time basis. Conse¬ 
quently, the Project will continue to design and execute 
several experimental tests of the system indicators under 
real-time operating conditions. These tests will be based on 
preselected time periods to prevent ex post facto interpreta¬ 
tions and evaluations. The results of these tests will be 
analyzed; depending on their outcome, appropriate modifica¬ 
tions will be made to the system. 

Using selected features of the automated EWAMS as well 
as specially developed internal software, EWAMS has been under¬ 
going a series of real-time tests and evaluations. This pro¬ 
cess has involved extensive testing of the system as well as 
the inputs of expert judgments and the advice of current 
and past intelligence analysts and managers. 

This process should be continued and accelerated. Testing 
in a real-time mode is the absolutely indispensable pre¬ 
requisite for transferring a system that can reasonably be 
expected to be used on a day-to-day basis. Stech (1979) 
points out that explicit, rigorous evaluations of intelligence 
products and processes are rare and perilous because of 
bureaucratic politics and similar factors. In an off-line 
mode, however, there is a greater likelihood of employing the 
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results to enhance the performance and quality of the system. 
As in the past, actual rates of success and failure (hits, 
misses, false alarms, correct rejections) will be ascertained. 

The existing framework for real-time testing and evalua¬ 
tions is skeletal. It involves short-term retrospective 
"postdictions" and short-term prospective projections; 
comparisons of the performance of EWAMS vis-a-vis expert 
judgments have also been undertaken. Clarkson et al. (1980) 
and other sources can be employed during FY 81 to expand 
and refine the evaluation framework. For example, the 
use of EWAMS data for monitoring and warning can be examined 
in a larger context (e.g., in terms of attribute and other 
contextual data). The use of EWAMS for constructing scenarios 
(employing approaches outlined in Clarkson et al., 1980) could 
also be pursued. These and additional enhancements and exten¬ 
sions will be considered and, as appropriate and feasible, 
implemented. 

The addition of new data files, indicators, and scans 
as well as many other new enhancements will greatly increase 
the amount of information available to the user. Thus, it is 
critical that the system software continue to be developed as 
the information retrieval, monitoring, and forecasting 
capabilities of the system are expanded. 
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Every effort must be made to continue to enhance the 
flexibility of the system without sacrificing simplicity. 
Even as the system has rapidly grown over the last several 
months with the addition of several new analytic features 
and the automated module, users still only need to enter 
simple, brief input in order to operate the system. New 
enhancements to the system will continue to be dependent on 
feedback and suggestions from potential users. Many sug¬ 
gestions from potential users have already been integrated 
into the system. Recent enhancements which are subsumed 
under the rubric of user requests/suggestions include: 

• Regional text 

• Keyword search 

• Error diagnostic checks 

• Numeric input parameters 

• Special purpose regions 

• Textual output 

In addition to maintaining system flexibility and 
simplicity, software developments must be compatible with 
the hardware and software of both potential users and other 
computer-based products of DARPA/CTD. This objective will 
require that all future developments in software and hard¬ 
ware be closely coordinated with other CTD projects. 
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SECTION II 
INTRANATIONAL I&W 
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7.0 INTRODUCTION 


Virtually no systematic, rigorous efforts have been 
made to develop and test procedures, approaches, and 
methodologies for analyzing, tracking, and projecting 
events and situations within nations. Intranational phe¬ 
nomena, however, are critical components of the I&W analyt¬ 
ical process. 

7.1 Background 


Just as international politics has been the source of 
data bases, methodologies, and conceptual frameworks which 
have been transferred and adapted to international I&W con¬ 
texts, students of comparative politics have probed and 
attempted to analyze the dynamics of violence, stress, and 
change within political systems. Nor is this work restricted 
to the domains of data collection and descriptive profiling; 
increasingly sophisticated causal models—which have accounted 
for from one-third to four-fifths of the variance—have been 
constructed and tested in the realm of civil strife or mass 
political violence. Recently, a study explicitly concerned 
with the forecasting of protest and rebellion has been 
published (Gurr and Lichbach, 1979). 

The I&W analyst now has access to a data base which is 
large, relatively comprehensive, and sufficiently varied 
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substantively and temporally. Increasingly more sophisticated 
analytical strategies can be and have been applied. The data 
base-dependent developments in basic international and intra¬ 
national political research (e.g., Hopple, 1978; Singer and 
Wallace, 1979; Wilkenfeld et al., 1978, 1979, 1980) and in 
the applied sphere (e.g., Andriole and Young, 1977; Bell et 
al., 1978; Daly, 1978) have been both prolific and impressive. 

Perhaps the richest single data base in quantitative 
international politics is the World Event Interaction Survey 
or WEIS, which currently constitutes the empirical foundation 
for the Early Warning and Monitoring System. WEIS and other 
events data sets comprise only one type of data in the fields 
of international and comparative politics. Also pertinent 
are various aggregate data sets which contain political, 
military, economic, social, cultural, and demographic 
variables as well as scattered quantitatively-based content 
analysis, elite biographical, and interview data collections. 
Furthermore, most of these data sets are available to 
researchers through the Inter-University Consortium for 
Political and Social Research (ICPSR) at the University of 
Michigan; DARPA/CTD and other agencies and institutions have 
transferred data sets to ICPSR. 

In the domain of international political analysis, 
(external) crisis research has obviously emerged as a viable 
subfield. The proliferation of case studies, frameworks, 
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propositional inventories, panels at professional meetings, 
special issues of journals, and other signals demonstrate the 
validity of this assertion (see, e.g.. Hopple and Rossa, 1980) 
Unfortunately, comparable activity has not characterized the 
study of intranational crisis (or internal affairs generally). 

Recent inquiry, however, has at least generated an impres 
sive number of theoretical frameworks and empirical proposi¬ 
tions in the amorphous realms of "internal conflict" and 
"aggressive participation." Such research has obvious 
potential relevance to the study of indicators of intra¬ 
national behavior. 

7.2 Problem Statement 


Recent months have witnessed the proliferation of 
domestic strife and tension in sub-Saharan Africa. The up¬ 
heaval in Uganda in early 1979, the civil war in Chad, the 
recent coup in Liberia, and the continuing stress in the 
Union of South Africa and Namibia are among the most obvious 
of a constellation of intranational stresses and strains in 
Africa. Quite clearly, there is a need for an intranational 
monitoring and warning system for I&W analysts tasked with 
Africa. 


See especially: Feierabend et al. (1972); Gurr (1970); 
Hibbs (1973); Jackman (1978); Muller (1977; 1979); Snyder 
(1978); and the sources cited in Hopple (1978). 


170 








The undeniable pervasiveness of intranational I&W con¬ 


cerns is highlighted by statistics on conflict, violence, and 
war. One study points out that there were 380 conflicts in 
the world from 1946 to 1964; 85 percent of these were in¬ 
ternal in nature (Orlansky, 1970). Included in the 85 percent 
were coups, military revolutions, mutinies, civil disorders, 
and internal guerrilla and civil wars. This, it should be 
noted, excludes more moderate manifestations of societal 
unrest. 

An intranational crisis system would focus on several 
interrelated, pervasive problems: 

• The development, evaluation, and improvement 
of procedures for forecasting and monitoring 
a range of intranational crises 

• The identification and observation of quanti¬ 
tative indicators for crisis warning 

• The integration of quantitative indicators 

and methods into an interactive, user-oriented, 
computer-based intranational crisis early 
warning and monitoring system 

The problem of intranational crisis warning is a complex 
one. Further, it has not elicited the systematic treatment 
which has been accorded to international crisis analysis. The 
responsibility for I&W analysis of such phenomena as internal 
stress, intranational crisis, and societal instability is 
diffuse; the prevailing approaches are unsystematic, ad hoc, 
and primarily "intuitive." 
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It is clear that in order to improve warning time of 
intranational and international crises, technology for that 
purpose must be continually developed and tested. Further¬ 
more, if the technology is intended for actual use, it must 
be tailored to the preferences and needs of those individuals 
who are actual participants in the warning process. This 
realization permeates the proposed technical approach dis¬ 
cussed later in this document. 

7.3 Proposed Solution 

An intranational monitoring and forecasting system 
designed for transfer to the I&W community would feature two 
central thrusts: 

• One would focus on the nature and impact of 
internal stress and political change and 
would monitor and forecast such phenomena 
as: 

- coups 
internal war 

- elite repression 

- mass instability 

• The second emphasis would be the realm of 
potential, emerging, and actual linkages 
between internal stress and instability of 
various kinds and external and international 
affairs. 

The existing EWAMS monitors international affairs and 
forecasts serious conflict and crisis episodes in the arena 
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of world politics. The first thrust of the proposed, new 
I&W system is analogous, except that the concern is intra¬ 
national affairs and the anticipation and tracking of in¬ 
ternal violence and crises. 

The second driving objective of such an I&W tracking 
and forecasting system reflects a prominent theme of both 
policy analysis and basic research. Do leaders engage in 
war in order to deflect attention from internal problems and 
failures? More generally, is there a nexus between what goes 
on within a country and that actor's relations with others? 
The 1978-1979 Uganda-Tanzania crisis and war—coupled with 
the impact of the Idi Amin regime and the coup which occurred 
during and because of the Tanzanian incursion into Uganda— 
clearly illustrates the impact and dynamics of the potent 
and complex internal-external linkage process. 

The linkage syndrome may be manifested in at least 
three forms. Intranational stress may lead to foreign 
conflict (e.g., intranational conflict spilling across the 
country's borders and leading to interstate conflict). 

Foreign conflict (war) may eventually promote intra- 
societal stress (violence). In addition, intranational 
stress could ultimately lead to foreign intervention. It 
would thus be necessary to consider three distinct types 
of linkage models, as depicted on the following page: 






Intranational Stress^-Foreign Conflict 

Foreign Conflict (War) ►intranational Stress 

General Societal Weakness ►Intranational Disorder^Being a 

Target of 
Foreign In¬ 
tervention 

A system relevant to African intranational I&W would con¬ 
sist of four primary modules: 

• Data base 

• Indicators 

• Forecasting methodologies 

• Computer base 

The assembly of a data base depends initially upon the 
assessment and selection of sources and a survey of avail¬ 
able and relevant types of data . The latter, of course, 
is integrally related to the specification of indicators . 

To a considerable extent, this issue has been addressed 
in the past. Available cross-national intranational data 
have been assessed and, to an extent, assembled in earlier 
DARPA/CTD-supported work (Hopple, 1978); Africa-specific 
data and sources have been discussed and evaluated in 
several studies (e.g., Jackman and Boyd, 1979 and Morrison 
et al., 1972). 
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The data base should be as real-time as possible. This ‘ 

suggests using newspapers and/or news chronologies and FBIS 

to generate data on internal African affairs. Background i 

] 

information and such necessary data as an historical intra- 1 

national crisis file can be acquired from news chronologies ] 

and/or comparative data banks. j 


What type of information should go into the data base? 

In international affairs, the EWAMS is built around a core 
data base called the World Event Interaction Survey (WEIS). 
WEIS, as noted in Section I of this report, consists of 
event/interactions (cooperative and conflictual acts) 
exchanged between and among countries as reported in 
several major data sources. Analogously, an intranational 
Africa-relevant I&W system should attempt to capture major 
event/interactions within countries and classify the behavior 
type of the discrete verbal or physical action. Whenever 
an event occurs and there is an identifiable actor and 
target, the item can be coded in an "intranational WEIS" 
format. For example, if a guerrilla group attacks a govern¬ 
ment post, that could be appropriately coded, with each 
element being assigned a numeric code in terms of: 


I 


• Time 

• Actor/s 


IL. 



Event type 






• Target/s 

• Issue type 

A variety of indicators could be developed. These indi¬ 
cators would initially be patterned after the ones in EWAMS. 

For example, EWAMS includes indicators of volume (total 
international behavior, cooperation activity, and conflict 
activity) and variety (tension and complexity or uncertainty). 

A system of parallel intranational indicators can be conceived. 
These will be discussed in detail in Section 8.3.3. 

A range of potential forecasting methodologies is avail¬ 
able for the central task of an intranational I&W system, 
which is the estimation of the probability of an intra¬ 
national crisis in country x at time t+1. Among these are: 

• An array of objective forecasting techniques 

Using sophisticated time-series modeling 
techniques 

Using probabilistic forecasting 
methodologies 

• Subjectively-based forecasting techniques 

- Using a panel of recognized external 
experts 

- Allowing for the input of EUCOM/J-2 
analysts to revise and update proba¬ 
bilities 
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Finally, the system would have a computer base. The 
hardware configuration will be at DARPA/CTD's Demonstration 
and Development Facility (DDF), with a PDP 11/70 on-site 
and arrangements for the transfer of the data base, indica¬ 
tors, and forecasting methodologies to EUCOM/J-2 analysts 
with actual use via a Tektronix graphics terminal, a 4907 
File Manager, and hard copy units. The software for the 
system would be parallel to and compatible with the extant 
Early Warning and Monitoring System software. 
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MICROCOPY RESOLUTION TEST CHART 


8.0 SUMMARY OP RECENT RESEARCH 


8.1 Overview 


This section summarizes the indirectly relevant research 
on intranational I&W in Section 8.2 and reports the work 
completed so far on the intranational/Africa I&W prototype 
under development at IPPRC in Section 8.3. 

Intranational political change and stress continuously 
affect and shape foreign policy—both directly and indirectly. 
However, approaches for tracking and analyzing intranational 
affairs in a systematic fashion are almost nonexistent. In 
contrast, international Indications and Warning (I&W) is a 
fairly well developed area. In the domains of quantitative 
military and—increasingly—political indicators, inter¬ 
national I&W analysts have access to fairly large empirical 
data bases and reasonably productive quantitative indicator 
systems. Eventually, an intranational I&W analogue should 
be developed and implemented. . ^ 

8.2 Intranational Crisis Research 


8.2.1 Background . Some research has been conducted 
on intranational crises, although the focus has often been 
narrow (i.e., certain kinds of crises) and the entire range 
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of intranational affairs has rarely been the object of con¬ 
cern. 


The domain of intranational crisis is not a unitary or 
simple phenomenon. Internal turmoil and unrest, for example, 
can be clustered into two broad realms: governmental in¬ 
stability and societal unrest (Hopple et al., 1977). The 
first dimension subsumes instability events which are con¬ 
fined to the formal political system while the second con¬ 
sists of behaviors which involve the mass public. This bi¬ 
furcation implies that there may be a fundamental difference 
between intra-systemic (i.e., actions involving the political 
elite and perhaps a counter-elite) and extra-systemic 
violence and unrest (see also Hibbs, 1973: 7-8). One 

salient classificatory dimension for intrastate crises may 
therefore be the range of subnational actors involved (or 
the extent to which the mass public—or segments of it—is 
mobilized). 

In addition to the range of actors and groups and such 
other potential criteria as the time span and the range of 
issues (e.g., issue-specific versus "diffuse" crises), the 
type of issue per se emerges as a key criterion. Re¬ 
searchers in intranational political analysis (e.g., Dahl, 
1961; Lowi, 1967) and foreign policy analysis (e.g., Rosenau, 
1966; Zimmerman, 1973) have exhibited some awareness of the 
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impact of issue area. Economic crises (e.g., recessions 
and depressions) and political crises (e.g., constitutional 
crises, the appearance of fissures within ruling coalitions, 
protest resignations of key cabinet members, etc.) may pose 
different problems and suggest varying implications. 

Electoral crises—such as critical elections which signal 
impending party realignment—should also be considered. 

Other types of intrastate crises could be delineated 
(such as technological-environmental crises), but the pri¬ 
mary concern here is to emphasize the importance of typing 
internal crises on the basis of issue content. This vital 
research task has been neglected in the past. If political 
science lacks a rich typology of situations, as George et 
al. (1971: xiii) note, this criticism is applicable with 
special force to international and intranational crisis 
analysis. 

In fact, the degree of conceptual specification and 
empirical progress within prominent "cells" of the "typology" 
is disappointingly modest. While Morse (1972) argues 
convincingly that analysts should allocate more attention to 
the phenomenon of international economic indicators for 
monitoring international affairs and forecasting crises, 
the cross-national analysis of intrastate economic crises 
is both sparse and unsystematic. 
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Empirical research on intranational conflict (in terms 


of its major dimensions and especially with regard to the 
linkage between intranational and international conflict) 
is voluminous. A small sample of the major published 
studies includes Banks (1972), Gurr (1967, 1968a, 1968b, 

1970), Gurr and Bishop (1976), and Wilkenfeld (1973). 

8.2.2 Models of internal crisis and stress . The entire 
spectrum of determinants of intranational unrest and con¬ 
flict ranges from macrosocietal structural conditions to 
microscopic precipitants. Considerable work is available 
on such remote macrolevel or structural determinants of 
intrasocietal unrest as economic development. A major example 
is the Hibbs (1973) causal analysis of a variety of potential 
structural sources of mass political violence. Among the 
predictor variables which he employed are: 

• Social mobilization, government performance, 
and social welfare 

• Sociocultural differentiation (ethnolinguistic 
fractionalization, group discrimination, poli¬ 
tical separatism) 

• Political system characteristics (democratiza¬ 
tion, regime type, political influence of the 
communist and non-communist left) 

Other examples include Hudson (1970), who assesses the 
impact of "environmental" influences (social modernization and 
homogeneity of political culture) and institutionalization 
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(differentiation and durability of political structures) as 
well as Jacobson (1973a), who examines the structural level 


forces of structural complexity, coerciveness, and system 
performance. Both analyses are cross-sectional (circa 1965); 
Hudson (1970) examines 63 countries and Jacobson (1973a) 75. 

More intensive research has been conducted in the 
economic domain. In addition to the Hibbs (1973) study, 
Parvin (1973) estimates the effects of an array of economic 
indicators (cross-sectional data for 26 countries from dif¬ 
ferent regions), while Sigelman and Simpson (1977) present 
findings for personal income inequality and several other 
economic and sociocultural aggregate indicators (also 
cross-sectional, for a sample of 49 nations). 

A recent study by Jackman (1978) specifies and estimates 
a model of the structural determinants of coups d'etat for 
the new states of black Africa. The results, which apply 
to the 1960-1975 period, indicate that: social mobilization 
and the presence of a dominant ethnic group are both destabi¬ 
lizing; multipartyism per se is destabilizing; electoral 
turnout in the last election before independence is stabiliz¬ 
ing. These and other findings collectively account for over 
four-fifths of the variance in black African coups. 

Various relationships between the other two major pre¬ 
dictor realms—intermediate micro factors (social attributes) 
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and immediate micro phenomena (psychological attributes)— 
and intrasocietal conflict have been reported. In fact, the 
best known theoretical frameworks in the literature, such as 
the work of the Feierabends and Gurr (see Feierabend and 
Feierabend, 1966; Feierabend et al., 1972; Gurr, 1968a, 

1968b, 1970; Snyder, 1978) emphasize such psychological and 
social psychological concepts as frustration and relative 
deprivation. 

The possibility that there is a nexus between internal 
and external conflict behavior has been supported by intui¬ 
tively plausible reasoning and by sociological conflict 
theory. In empirical research, the relationship between 
intranational conflict behavior and foreign conflict behavior 
has been examined from a variety of methodological perspec¬ 
tives (e.g., Hazlewood, 1973, 1975; Rummel, 1963; Tanter, 
1966; Wilkenfeld, 1973; Zinnes and Wilkenfeld, 1972). The 
linkage has also been measured in varying regional arenas 
and cross-national contexts (e.g., Burrowes and Spector, 

1973; Collins, 1973; Liao, 1976; Stohl, 1975; Wilkenfeld, 
1975; Wilkenfeld et al., 1972). 

The early work (Rummel, 1963; Tanter, 1966) showed that 
there is virtually no relationship between internal and 
foreign conflict. On the basis of historical data, Denton 
(1966) concluded that there is a positive relationship. 
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Wilkenfeld (1968) demonstrated that the relationship is 
moderately strong when type of state is taken into account. 

More recently, Kegley et al. (1978) reaffirmed the 
importance of considering the intervening variable cluster 
of type of nation. In a test using militarization (military 
expenditures as a percentage of GNP), the correlation with 
internal conflict (Gurr's civil strife data for 1961 to 
1965) is .15 and the correlation with foreign conflict (WEIS 
data, 1966 to 1969) is .24. When regimes are classified by 
level of military expenditure, the following results emerge: 

• There is a small positive relationship between 
domestic and foreign conflict for nations in 
the low militarization category. 

• There is almost no relationship for countries 
in the medium category. 

• There is an inverse relationship of considerable 
magnitude (r = -.49) for countries in the high 
militarization group (Middle East and the 
Communist nations). 

• The large majority of the cases fall into the 
first two categories. 

Some attention has also been given to the question of 
the linkage between war and intrasocietal political vio¬ 
lence. Stohl (1975) uses an interrupted time series quasi- 
experimental design to examine the war involvement-intra- 
societal level changes-intrastate conflict sequences. The 
dependent variables are the type and extent of internal 








political violence; the introduction of war leads to a 
search for differences between pre- and post-test slopes and 
intercepts. For all five major U.S. wars since the 1890s, 
Stohl discovers that war exerts a significant (but 
substantively variable) impact on patterns of internal 
unrest and violence. 

There have been various causal models constructed and 
tested in the cross-national study of civil strife or what 
Hibbs (1973) refers to as "mass political violence." The 
system-level studies have all relied on time series aggregate 
data banks, which have been amassed for all or most countries 
of the world for periods of a decade or more. The models 
have been both universal and regional in scope; both recur¬ 
sive and non-recursive models have been developed, a variety 
of variable clusters and manifest indicators have been 
included, and the deductive or theoretical route to model- 
construction has been surprisingly well represented (especi¬ 
ally in the work of Ted Robert Gurr; see, e.g., Gurr, 1970; 
Gurr and Duvall, 1973; and Gurr and Lichbach, 1979). 

The empirical results have been respectable and, in many 
instances, impressive. As noted, Jackman (1978), in a 
regression model which measured the impact of structural 
determinants on coups in Africa from 1960-1975, accounted for 
over four-fifths of the variance. Hibbs (1973) , in two cross- 
sectional analyses (1948-57 and 1958-67) of 108 countries. 
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reported findings for a comprehensive series of single equa¬ 
tion hypotheses and for a "block recursive" multiequation 
model of mass political violence. The core of the "final" 
Hibbs model is reproduced here as Figure 8-1. Gurr and 
Duvall (1973), in a study of 86 cases for the 1961-1965 
period, constructed and tested an 11-variable, bloc recur¬ 
sive (i.e., non-recursive) simultaneous equation causal model; 
the for the dependent variable magnitude of political con¬ 
flict (extent and intensity of conflict) was almost .75. 

Figure 8-1 

THE CORE OF THE FINAL HIBBS 
MODEL OF MASS POLITICAL VIOLENCE 



More recently, Gurr and Lichbach (1979) have presented 
results for a more sophisticated causal model of domestic 
conflict, again using the 86 nations and the 1961-1965 data. 
The analysis involved six separate sets of simultaneous 
equations (extent and intensity of protest, external dissident 












support, external regime support, extent and intensity of 
rebellion). The 's ranged from .36 (intensity of protest) 
to .81 (intensity of rebellion), with rebellion accounted 
for especially well (.81, as noted, and .68 for extent or 
man-days of participation). 

Several characteristics distinguish the Gurr and 
Lichbach (1979) model. First, the level and extent of 
theoretical specification are both encouraging—and espe¬ 
cially as rebuttals to critics who have excoriated quantita¬ 
tive comparativists for "barefoot" empiricism. Conflict is 
viewed as a continuous process rather than as a set of 
discrete events; the central focus are the clusters of 
conditions which determine the ebbs and flows in the scope 
of action by anti-regime forces and the intensity of the 
regime's response. The theoretical system is coherently and 
comprehensively articulated. 

Secondly, Gurr and Lichbach (1979) devote considerable 
attention to the problem of empirically generating composite 
measures of their theoretical concepts. Essentially, they 
pursue an elaborate four-stage process for weighting the 
measures in each index. The methodology, which combines z- 
scoring with regression analysis, is analogous to but more 
involved than the latent variable construction process in 
Partial Least Squares (PLS) causal modeling (see Wilkenfeld 
et al., 1979, 1980). 


187 










The third noteworthy feature is the explicit use of the 
model for forecasting purposes. With data primarily from 
1970 for the causal variables and 1971-1975 conflict data 
and a quota sample of 10 cases (i.e., countries) from the 
larger group of 86 nations, the techniques were used to 
forecast properties of conflict in the 1970s. Forecasted 
and real data for 1971-1975 were then compared. Not surpris¬ 
ingly there were a number of errors; however, rebellion was 
predicted fairly well and the equations for intensity (deaths) 
showed a better fit than the ones for extent (man-days of 
participation). 

As this review of models of intranational crisis sug¬ 
gests, the phenomenon is not in principle unpredictable 
(see, e.g., Jackman, 1978). The problem is that the research 
has focused on the longer-term, more slowly developing 
factors. More dynamic phenomena, which are more relevant 
from the perspective of an I&W analyst with short-term and 
real-time concerns, have been neglected. The proposed tech¬ 
nical approach in Section 8.3 will address this need. 

8.3 A Prototype Intranational Warning and Monitoring System 

The promising and successful development of the DARPA 
Early Warning and Monitoring System (EWAMS) for tracking and 
forecasting inter state crises has led to increased interest 
in the development of a prototype system for analyzing 
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intran ational political behavior. This interest has mater¬ 
ialized in the form of a feasibility study now underway, an 
effort being pursued by several contractors who have been 
closely associated with EWAMS. While this feasibility study 
is not yet completed, it is important at this point to 
delineate the next logical steps in the development of an 
intrastate crisis warning and monitoring system. 

The feasibility study addresses only a limited set of 
questions; these are designed to contribute to a preliminary 
assessment of the utility of collecting intranational events 
data. The questions include: 


• Is the process of coding intranational events 
feasible—can intranational events be collected 
as routinely as international events? 

• Is the payoff, in terms of the number and nature 
of intranational events for specific countries, 
sufficient for conducting subsequent analyses? 

• What are the relative payoffs of the various 
event sources: newspapers; chronologies; FBIS; 
and cables? 

• How do the EWAMS indicators (e.g., z-scores, 
tension, etc.) perform in an intranational 
context? 


The Africa Warning and Monitoring System (AWAMS) was 
developed as a prototype during FY 80. The sequential and 
interrelated stages of this effort included: 

• A conceptual blueprint and a set of goals 
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• An information base (data) design 

• An indicator design 

• A software design 

8.3.1 The blueprint . An intranational I&W system 
tailored to a short-term time horizon would provide a detailed 
scheme for monitoring and forecasting intranational affairs. 
The system would have an historical base and would operate 
in a real-time mode. The goals of comprehensive and system¬ 
atic information storage, retrieval, and display, current 
tracking, and anticipation of future situations and crises 
would all drive the development and testing of the system. 

Operationally, the system would have five basic fea¬ 
tures. In summary form, these are: 

• User-oriented in nature 

• Parallel to the existing EWAMS in design 

and operation 

• Systematic and comprehensive in scope and depth 

- A wide range of indicators 

- A variety of forecasting methodologies 

Coverage of all major potential intranational 
crisis arenas and stress contexts 

• Short-term and real-time in focus and nature 

• Cumulative 

- Patterned after EWAMS 
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Builds on existing data bases, models and 
conceptualizations in basic cross-national 
research (extensive work by Banks, Gurr, 
Jackman, and many others) 


An initial evaluation of sources for Africa by IPPRC 
showed that several distinct types exist. Among these are: 


• Prestige and other newspapers 

Internationally known newspapers such as 
the New York Times and the Times of London 

Africa newspapers such as the Daily Graphic 
of Ghana 

- News wires (UPI, Reuters, etc.) 

• News chronologies and almanacs 

- Such general (worldwide) sources as Facts on 
File , Deadline Data on World Affairs , or 
Keesing^s Contemporary Archives 

Such African sources as Africa Diary , African 
Digest , and Africa Yearbook and Who's Who 

• Expert-generated data (i.e., panels of area and 
country experts from academia and government for¬ 
eign affairs, national security, and intelligence 
analysts) 

• Foreign Broadcast Information Service (FBIS) and/or 
classified sources 

• Cross-national data banks 

- Arthur Banks of the State University of New 
York at Binghamton (domestic conflict indicators 
all countries in the world, 1800s to about 1979) 

Ted Robert Gurr of Northwestern (civil strife 
indicators, all countries, 1950s-1960s) 

Robert Jackman of Michigan State (coup data 
for 30 African countries, 1960-1975) 
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D.G. Morrison and others, Black Africa: A 
Comparative Handbook (a 1972 source with com¬ 
parative statistical profiles in every conceiv¬ 
able area—ranging from ecology and demography 
to political instability—as well as country- 
specific profiles) 


Indicators for monitoring and forecasting intranational 
affairs are virtually infinite in scope, nature, and purpose. 
A very partial but representative list follows: 


• Volume 

Total intranational behavior 

- Total cooperative behavior 

- Total conflictual behavior 

• Variety 

Intranational stress or tension (an intranational 
"thermometer") 

Issue focus or complexity (dominance of one, 
a few, or many substantive issues) 

• Frequency versus intensity of intranational stress 

General stress 
Specific forms of stress 

• Change index (trends over time in intranational and 
international economic health, social policy, etc.) 

• Polarization 

Intra-elite 
Competing elites 
Elite/mass 


• System performance 

• Demand on the system 
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• Support for the system 


The relevance of the discriminant methodology, which has 
recently been applied to EWAMS indicators with considerable 
success (see Section 4.0), will be ascertained. The pool of 
indicators must be selected and relevant thresholds must be 
established. Single and multiple indicator strategies will 
be tested. Issues relating to hit, false alarm, correct 
rejection, and miss rates will be central in the assessment 
of the forecasting component of the system. 

The Africa Warning and Monitoring System (AWAMS) 
prototype has initially featured the tracking or monitoring 
dimension. The forecasting goal will be pursued after it 
has been established that the system successfully monitors 
ongoing activity. 

In designing and implementing AWAMS, prior research (see, 
e.g.. Section 8.2) and relevant approaches, designs, and frame¬ 
works have been surveyed. Integral to the prototype effort is 
the concept of an event/interaction. An evant/interaction 
is a nonroutine and temporally delimited situation which 
features an actor (or subnational actor), a target (or sub¬ 
national target), an event type, and a time frame. The event/ 
interaction framework provides the basis for designing, 
developing, and testing the system. 
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In implementing such a framework, it is necessary to 
consider and explicitly take into account the salient 
differences between international and intranational politics. 
For example, the units of analysis in international politics 
are generally nation-states or their elites. Thus, WEIS 
includes only events that are public, nonroutine (which 
differentiates a discrete event from a transaction), and 
‘ official. Even in international politics, however, other 

actors can be relevant; these range from subnational actors 
(e.g., liberation or secessionist forces) to supranational 
actors (e.g., universal or regional organizations). 

The domestic political landscape is even more complex. 
Even a relatively simple model, such as the one displayed here 
as Figure 8-2, features four distinct types of actor groups in 
intranational politics. The following kinds of linkages must 
be considered in developing a realistic, valid coding scheme; 

• Elite ► Counterelite 

• Elite ► Mass 

• Counterelite ►'Mass 

• Mass ^ ► Mass (e.g., conflict between different 
tribes or religious groups, interest group inter¬ 
action, etc.) 

More sophisticated typologies could also be introduced (e.g., 
the elite, other officials, the elite public, the attentive 
public, apoliticals, etc.). 
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Figure 8-2 

TYPES OF ACTORS IN AN INTRANATIONAL I&W SYSTEM 


ACTORS’ ORIENTATION 


ATTITUDE 

TOWARD 

REGIME 


MASS ELITE 


PRO-REGIME 

MASS-ORIENTED 

PRO-REGIME 

ELITE-ORIENTED 

ANTI-REGIME 

MASS-ORIENTED 

ANTI-REGIME 

ELITE-ORIENTED 


Our initial research on the realm of African internal 
I&W thus reviewed the rather extensive—albeit scattered— 
basic research literature in this area. In addition, rele¬ 
vant R&D work on monitoring internal affairs provided 
crucial evidence that a systematic approach to tracking 

developments within countries is both feasible and poten- 
* 

tially productive. These two streams both flow into the 
"Intranational Politics" arrow displayed in Figure 8-3. 

The other major influence shown in Figure 8-3 is the 
international politics-oriented EWAMS. The concept of an 
event applies to both international and intranational 
politics. A modified EWAMS event coding scheme was developed 
for collecting data on internal affairs. 

*See Slater and Orloski (1978). 
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8.3.2 Data collection design . The key feature of the 


internal event data scheme consists of the actor-target 
concepts. In intranational political analysis, there are 
subnational actors (senders within countries) and sub¬ 
national targets (recipients within .countries). An example 
would be a group within the general or mass public (e.g., 
an isolated gathering) which protests against a policy 
promulgated by an actor within the government (e.g., the 
head of state). 

The AWAMS subnational actors and targets correspond 
to EWAMS countries and other actors. There is an AWAMS 
(internal politics) event category scheme as well as one 
for EWAMS (international politics). Numerical codes are 
assigned to all of these discrete items. Figure 8-4 
portrays the AWAMS data collection process. 

The well known government/society dichotomy is an 
excellent conceptual beginning for delineating actors and 
targets within states because the relationship between the 
rulers and rul-ed is central to the Analysis of interstate 
conflict and stability in all countries. Other relation¬ 
ships are also important: intra-governmental and intra- 
societal patterns may indicate tranquility or crisis for 
the polity as a whole. Therefore, a somewhat more elaborate 
conceptual division of political actors is desirable. 

Table 8-1 offers one such actor/target list. 
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The government and military are separated in this 
scheme in order to emphasize the role of the military in 
society. Each is subdivided into sub-actors as well in order 
to permit greater flexibility and provide for detail within 
the general scheme. Government is divided into the head of 
state and executive, legislative, and judicial components, 
as well as regional/local governments. The military 
includes the army, air force, navy, and domestic security 
components. 

The ruling political party is separated from the govern¬ 
ment because this relationship is not always one of identity; 
indeed, the ruling party's relationship to the government 
(or one component therein) may be a source of conflict or 
instability, as is the case regarding intra-party relation¬ 
ships (factions, leadership/elites, membership interactions). 
Opposition political parties are similarly important and 
identifiable actors in intrastate politics; because a party 
is an organization whose chief avowed aim is the placement 
of its members into government positions, parties are unique 
actors in virtually all political systems. 

The media play a special role in intrastate politics 
since they serve as communications channels among other 
actors and also interact politically on various issues. 

The structural arrangement linking the media to the govern¬ 
ment, parties, and other actors varies greatly across systems 
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and it is important to distinguish among such arrangements 
(independent, government-operated, party-operated). 

The mass of the political system refers to the generally 
unorganized or informally organized collections of indivi¬ 
duals. "Masses" may refer to the public in general, to 
scattered collections or gatherings across the nation, to 
an isolated grouping, or to a specific region-wide or local¬ 
wide collectivity (see Table 8-2). Students and dissidents 
are recognized as special cases because of their tradition 
as especially outspoken elements of the political "mass." 

Table 8-2 

TYPES OF MASS ACTORS 


PERMEATION UNDIFFERENTIATED GROUPING 

OF 

CITIZENRY DISTINGUISHABLE GROUPING 


The "society" is comprised of parties, media, masses, and 
relevant interest groups and organizations (Slater and Orloski, 
1978: 10). Often, interest groupings can be simply defined 

as part of the masses; however, organized (or semi-organized) 
masses should be recognized as special actors in political 
systems because they are both distinct in their relative 
capabilities and dealings with other actors and potent 


TERRITORIAL COVERAGE 
NATION-WIDE LIMITED DOMAIN 


GENERAL 

REGIONAL/LOCAL 

SCATTERED 

ISOLATED 
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(because of organization) in their abilities to cause change. 
Labor (union and non-union) and business (of various kinds) 
have a very important relationship with each other and with 
other political actors and often exert considerable organi¬ 
zational power. Socio-cultural groups (religion, race, 
ethnic/tribal) are also important and often organized seg¬ 
ments of the population, normally exerting pressures regard¬ 
ing distributional policies (of power or wealth) which affect 
basic national unity; separation of race, religion, and 
ethnic/tribal groups is theoretically difficult but, in 
practice, quite feasible and desirable (e.g., racial versus 
tribal conflict in Africa). 

Table 8-3 offers a modified WEIS events coding scheme 
for intrastate analysis. It draws upon the Slater/Orloski 
work, as well as various conflict event studies and compara¬ 
tive research conceptualizations, in order to include all 
relevant event-types within the broad categories. The latter 
are identical to the WEIS scheme with one minor exception: 
"expel" has been replaced by "ban," a more inclusive form 
of activity of this genre. Thus, the coding scheme is 
virtually identical to the WEIS scheme (facilitating coder 
training and compatibility/comparability in analyses) but 
includes as sub-categories the behavioral forms which are 
distinctively intrastate. 
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Table 8-3 


EVENT CODING SCHEME FOR INTRASTATE DATA 


1. YIELD 

011 Submit to penalty or arrast 
012 Return property, retreat 
013 Seek accommodation with 
014 Admit wrongdoing, retract 
015 Cede power to 

2. COMMENT 

021 Decline to comment 
022 Comment on situation 
023 Explain situation or rights 
024 Explain policy or feelings 
025 Announce, provide data 
026 Fill or change position in 
(by appointment, election) 

027 Change rules or charter of 

3. CONSULT 

031 Meet with, host or visit 
032 Communicate with, negotiate 
033 Vote, hold election or meeting 

4. APPROVE 

041 Praise, honor, condolences 
042 Endorse or support 
043 Rally in support of 

5. PROMISE 

051 Promise support or aid 
052 Assure or reassure 
053 Promise reform or rights 

6. GRANT 

061 Apologize, express regret 

062 Invite, give asylum 

063 Grant rights, recognition 

084 Release captive or prisoner 

085 Suspend negative sanctions 

066 Grant position to (by appointment, election) 

7. REWARD 

071 Give money or material aid 
072 Give assistance or other aid 

8. AGREE 

061 Agree on issue or contract 
082 Agree to future cooperation 
083 Ally with 
084 Merge with 

9. REQUEST 

091 Request information 
092 Request support or aid 
093 Request action or change 
094 Request rights or membership 

10. PROPOSE 

101 Offer soecific proposal 

102 Urge or suggest 


11. REJECT 

111 Reject proposal, demand, or threat 

112 Refuse to allow, oppose 

113 Defy law, rule, command 

12. ACCUSE 

121 Charge, blame, criticize 

122 Oenounce, denigrate 

123 Investigate 

13. PROTEST 

131 Complain, protest verbally 

132 Symbolic act of protest 

14. DENY 

141 Deny attribution 

142 Deny accusation or claim 

15. DEMAND 

151 Issue command, insist upon 
action or change 

152 Claim rights, insist upon 
obligations 

16 WARN 

161 Warn against act, policy 

162 Warn of impending danger 

17. THREATEN 

171 Threaten, unspecific 

172 Threaten, specific sanctions 

173 Ultimatum (time limitation) 

18. DEMONSTRATE 

181 Protest demonstration 

182 Mobilize, move armed force 

19. NEGATIVE SANCTIONS 

191 Cancel, postpone, halt 

192 Reduce or suspend aid or support 

193 Strike; harass; impose penalty 

194 Censor, reduce rights or access 

195 Resign from; secede from 

20. BAN 

201 Exile, deport, fire, lock-out 

202 Close down, dissolve, purge, expel 

21. SEIZE 

211 Seize possessions, position 

212 Arrest, detain, kidnap 

22. FORCE 

221 Riot 

222 Destroy, demolish, bomb (non-injury) 

223 Assassinate, torture, execute 

224 Armed engagement or 
injurious bombing 

225 Coup, attempt coup 
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In order to establish the comprehensiveness of the event 
coding scheme, several major non-directional events —largely 
conflictual—can be compared to their counterparts in 
Table 8-3: 


• Coups : a force act (225); actor may be military 

(200) ; target normally head of state (110). 

• Constitutional Change ; change rules (027); actor 
may be mass (500) through vote (033) regarding 
government (100). 

• Executive Change : in election, mass (500) grants 
position (066) to a party leader (310 or 410) and 
appoints (026) head of state (110). 

• Cabinet Change : usually the head of state (110) 
appoints (026) to executive (120); often also 
grants position (066) to some group (e.g., labor 
official 1710]). 

• Purge : usually the head of state (110) fires 

(201) executive official (120) or party leaders 
(310). 


8.3.3 Indicator design . The prototype AWAMS indicators 
are patterned on those available to the user of the inter¬ 
national EWAMS. Thus, internal affairs may be monitored on 
a daily, weekly, or monthly basis in terms of: 


• Total activity 

• Conflictual activity 

• Cooperative activity 

• Tension 

• Uncertainty or complexity of relations 
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One potentially important change in the computation of 
these indicators should be noted. Whereas in international 
interactions events which are "comments" represent a form of 
cooperation (communication which is not hostile is generally 
viewed as a sign of cooperation), this is not necessarily 
true of comments exchanged within a country. The intrastate 
coding scheme intentionally codes all "neutral" events in 
one category (comment) and includes such actions as announce¬ 
ments and the filling of administrative vacancies. Such 
"neutral" events should be excluded from the cooperation 
activity indicator; they represent very ambiguous actions. 
However, the importance of these events as signals of 
activity recommends that they be included in the total 
activity indicator computation procedure. 

The concept of progressive z-scores can also be trans¬ 
ferred from interstate to intrastate indications develop¬ 
ment. This scoring procedure can be applied to any indi¬ 
cator in order to transform a "snapshot" portrayal into an 
historically grounded description of current events. By 
providing a measure of an indicator readings's unusualness 
when compared to prior readings, progressive z-scores 
will reflect the extent of change in interaction processes 
within a country, thereby standardizing indicators to 
country-specific contexts. Because differences across 
'$■ ’ , political structures are vast, this contextualization is 

ot great importance in intrastate indicator development; 
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unfortunately, the utility of this indicator approach is 
severely limited for the near future because of a lack of 
retrospective data (upon which z-scores depend) in many 
event categories. 

Other indicators of intrastate events may be culled 
from the especially rich format of the current data col¬ 
lection effort. Especially relevant is the repression 
index developed by Slater and Orloski (1978). By totaling 
the frequencies of events which are coded as repressive in 
the data base, the analyst is provided with an indicator 
which has been directly related to crisis warning. 

A second feature of the data base is the "issue" code, 
which provides a context for the event descriptions (Slater 
and Orloski, 1978). This information can be employed in 
two ways to provide additional indications of intrastate 
activities. 

First, the intrastate analyst should be provided with 
the option of selecting one or several issue contexts for 
delimiting the range of events to be considered; for example, 
the indicators of repression, conflict, tension, etc., can 
reference events concerning political issues, or economic 
issues, or any other set of issues rather than all issue- 
contexts combined. In this manner, the analyst may isolate 
•the general problems which underlie observed event indicators. 
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The second use of "issue” information is the application 
of measures of variation (e.g., Hrel) to assess the degree to 
which activity is concentrated in a small number of issue 
categories or diffused across all issues; a plausible hypoth¬ 
esis regarding this type of indicator is that both extreme 
concentration and extreme diffuseness are indicative of gov¬ 
ernmental instability, while moderate concentration/diffuse¬ 
ness is normal in political communications. 

8.3.4 Prototype data base . In addition to the general 
framework, the data collection design, and the indicator 
design and the preliminary specification of indicators, em¬ 
pirical data are also being amassed. These data are for 
intranational affairs within and international relations 
involving sub-Saharan African countries. 

The question of sources is central to the data base 
component of the prototype system. Figure 8-5 depicts the 
current and potential sources for the intranational (Africa) 
system, the international (Africa) system, and the 
international (global) system. Currently, internal event 
data are amassed on a continuous and real-time basis from 
MAG/TOL (Manchester Guardian , Times of London), classified 
(cable traffic) data, and FBIS (the Foreign Broadcast 

* 

The international (global) system refers to EWAMS, 
which includes international data on Africa as part of 
the overall system. 
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Figure 8-5 

data SOURCES POR AWAMS AND EWANS 



global 






Information Service) data. Several months of Reuters data 
for African internal politics and interstate relations have 
also been collected in a pilot study. 

8.3.5 Software . The software design and implementa¬ 
tion for the prototype are proceeding on schedule. Thus 
far, there have been two noteworthy software accomplishments. 
First, the data entry software used for WEIS/EWAMS data has 
been revised to accommodate simultaneous intra- and inter¬ 
national data entry. The original GEN software now exists 
as the modified GEN 2 software and is being used by IPPRC 
and has also been transferred to other DARPA/CTD contractors. 

Secondly, the prototype AWAMS can be analyzed via 
software. Thus, the user may analyze subnational actors 
and targets for any country in the system; data may be 
displayed in tabular and graphic formats and textual re¬ 
trievals are possible. 
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9.0 FUTURE RESEARCH 


The prototype AWAMS (Africa Warning and Monitoring Sys- 
stem) currently features two sources of data: MAG/TOL (Man¬ 
chester Guardian/ London Times ) and FBIS (Foreign Broadcast In- 
* 

formation Service). The data cover the period from 800101 to 
the present. Future research directions will involve: 

• Adding classified data 

• Enhancing the prototype system 

• Testing the data and indicators 

• Devising and implementing an analytic 
strategy for intranational crisis warning 

9.1 Source Performance 


Our research on international politics suggests unequivo¬ 
cally that dual or multiple sources are both advisable and 
necessary. No data source is "perfect" or foolproof. To 
maximize hits and minimize misses and false alarms, it is 
crucial that data profiles be constructed from two (or more) 
sources of information. 

Our preliminary comparisons of MAG/TOL and FBIS data 
on African internal affairs reinforce this conclusion. 

In addition, Mathtech, Incorporated is collecting and 
analyzing cable traffic data (also under DARPA contract). 
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While it would be premature to offer firm generalizations, it 
is clear that the two sources together are better than either 
alone. FBIS seems to provide a more detailed portrait and 
therefore may be preferable for daily or weekly tracking; how¬ 
ever, MAG/TOL yields patterns of interest and often confirms 
or complements FBIS. In other instances, MAG/TOL offers a 
unique picture of internal affairs in a given sub-Saharan 
African country. 

Recently, software has been developed which permits the 
simultaneous graphic display of AWAMS data and indicators 
for any two sources. This feature will be of value for our 
future work on comparative source performance. 

9.2 Analytic Strategies 

Quite a few potential analytic strategies are mapped 
out in Hopple and Slater (1980). These range from approaches 
for describing patterns in the data to methodologies for 
weighting, scaling, and otherwise transforming the raw data. 
Equally central will be the selection of a strategy for 
generating probabilistic internal crisis judgments. 

* 

Any two AWAMS or EWAMS sources may be so juxtaposed; 
refer to Figure 8-5 in the preceding section. The software 
was developed by Mark Waddell of the EWAMS computer science 
staff. 
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A problem with which event researchers have often been 


confronted is the selection of time periods for analysis. 

This has resulted in a greater emphasis on the use of weekly 
or daily indicators in EWAMS, as opposed to the more tradi¬ 
tional monthly aggregations. The time period problem will 
become more complex as analysis of intranational events 
proceeds, since it is highly likely that such events will 
unfold in a far less systematic pattern—consequently, less 
conducive to monthly and possibly weekly analysis. The use 
of moving averages is suggested as a way to smooth the event 
pattern so that each day adds to our knowledge of the pre¬ 
vious pattern without disturbing that pattern and without 
forcing the analysis into arbitrary and artificial intervals 
of time. Thus, in subsequent stages of the effort, moving 
averages (combined with some form of weighting such as 
exponential smoothing) should be incorporated into an intra¬ 
national monitoring and warning system. 

Scaling attempts to go beyond events viewed as simple 
frequency counts; weighting is designed to rectify the fre¬ 
quent criticism that the assignment of equal weights to event 
categories is both arbitrary and unjustified. In the analysis 
of intranational data, there have been numerous scaling 
studies. 

Often, this has involved the use of factor analysis in 
an attempt to reduce discrete indicators to a smaller set 
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of more primordial dimensions. Prior research in this area 
has unearthed the intranational war versus the societal unrest 
or turmoil dimensions. Elite versus mass instability factors 
(e.g., coups versus mass political violence) have also fre¬ 
quently been inferred. 

There are quite a few data-reduction and/or scaling 
techniques (factor analysis, Guttman scaling, etc.). The 
method of additive clustering or ADCLUS (see Shepard and 
Arabie, 1979) is a fairly recent alternative to the inher¬ 
ently continuous spatial models of multidimensional scaling 
and factor analysis and the strictly hierarchical models of 
discrete clustering. Clearly, intranational event data are 
neither continuous nor necessarily hierarchical in nature. 
ADCLUS subsumes hierarchical clustering as a special case and 
can be regarded as a discrete analogue of principal compo¬ 
nents analysis. 

ADCLUS and alternative clustering approaches are 
designed to represent a set of discrete categories as a 
smaller collection of subsets. At the same time, clustering 
avoids the unfortunate tendency in political research to 
assume unidimensionality (e.g., the conflict-cooperation 
continuum). 

Employing the semantic differential attitude measure¬ 
ment methodology, Calhoun (1971) transformed the WEIS 
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(World Event Interaction Survey) international event data 
categories into weighted indicators . As he contends, 
unweighted data often lead to the overemphasis of certain 
categories and the underemphasis of others. Expert judges 
were used to evaluate the basic WEIS event concepts across 
41 standard bipolar (e.g., good-bad, strong-weak) adjective 
scales. 

A similar approach to assigning weights to intra¬ 
national event concepts could prove to be very productive. 
Vector geometry can be employed to describe event concepts 
in a spatial context. From the semantic differential judg¬ 
ments a variety of ways to rank and score event categories 
can be applied (for example, the absolute distance of the 
concepts from the origin of the semantic space, evaluation 
x potency, etc.). 

The ADCLUS scaling and the semantic differential weight¬ 
ing approaches represent only two of many potential scaling 
and weighting options. In testing intranational indicators, 
a variety of candidate approaches could be experimented 
with. 


A viable intranational I&W tracking system should build 
on the event data base of the prototype. Various cognate 
data bases can be considered; these will be discussed below. 





Some of these are designed to facilitate the charting of 
more global trends, situations, conditions, states of affairs, 
and contexts. 


"Objective" data (including event data) are sometimes 
referred to as "hard" data—in contrast with "soft" (expert- 
based or judgmental) data. Popular in past decades, expert- 
derived data were eclipsed by objective data in the 1960s 
and 1970s. In recent years, however, there has been a 
"renaissance" in the area of expert data (see Hopple and 
Kuhlman, 1980). The trend is toward the meshing of objec¬ 
tive and subjective data. 

From a purely pragmatic perspective, this trend is 
laudable because users of computer-based and other data 
systems are generally suspicious of hard data and comfort¬ 
able with expert assessments. For successful transfers to 
the "user community," then, the inclusion of expert-based 
data can be expected to facilitate the process. 

Interestingly, the use of judgmental data also attracts 
support from the vantage point of less applied research 
criteria. As a recent GAO (1980) critique of defense R&D 
concludes, "squishy" problems cannot be reduced to or 
handled completely by hard models; the assessment concluded 
that judgments should be incorporated into and synthesized 
with models. This implies that consideration should be 
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given to the enhancement of the prototype intranational data 
by incorporating subjective judgments. 


Experts have always played a central role in academic 
international affairs and in applied national security and 
defense settings. Country and area specialists comprise a 
large cadre in the international and comparative politics 
research communities; State, DIA, CIA, and other components 
of the intelligence and national security analysis complex 
are often structured in terms of country desks and regional 
foci. 


The burgeoning literature on political risk assessment 
within and for private corporations provides a backdrop and 
a point of departure for this discussion. Almost all of the 
extant political risk analysis frameworks and services rely 
on country and area experts, who may be academicians, con¬ 
sultants, retired Foreign Service or CIA personnel, or cor¬ 
porate executives with extensive experience abroad (see 
Haendel, 1979; Slater and Longo, 1980). 

Whether used in an ad hoc or structured fashion (e.g., 
simply eliciting judgments, perhaps via a questionnaire and 
in terms of a checklist of factors, vs. Bayesian-based 
approaches), the political risk analyst relies on expertise. 
Although other approaches are used—including social scientific 
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techniques—the expert is the preferred and dominant "method¬ 
ology" (Kobrin et al., 1980). 

An apparent anomaly surfaces, however, when extensive 
evidence from psychology is introduced. An avalanche of data 
from behavioral decision theory research demonstrates that 
people exhibit a number of errors, biases, and fallacies when 
they evaluate evidence and make judgments. Furthermore, 
statistical models generally perform better than intuitive 
judgments—at least in certain areas. 

Massive supportive evidence for the latter proposition 
was catalogued in Paul Meehl's 1954 book Clinical Versus 
Statistical Prediction: A Theoretical Analysis and A Review 
of the Evidence ; Meehl reported that a proper linear model 
(e.g., simple regression analysis, discriminant function 
analysis) predicts a numerical criterion from numerical pre¬ 
dictor variables more accurately than skilled experts. More 
recently, Dawes (1979) presents evidence that the same holds 
for an improper linear model (i.e., one in which the weights 
are chosen by some nonoptimal method—equality, intuitive 
weights, etc.). As Dawes (1979: 573) emphasized, however: 

The statistical model may integrate the 
information in an optimal manner, but it 
is always the individual (judge, clinician, 
subjects) who choose variables. Moreover, 
it is the human judge who knows the direc¬ 
tional relationship between the predictor 
variables and the criterion of interest. . . 
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Dawes (1979) refers to bootstrapping as the process of 
building a proper linear model of an expert's judgments about 
an outcome criterion and then using the linear model rather 
than the judge. In a number of studies, bootstrapping has 
worked. Random linear models perform about as well as the 
judge-based models; the equal weighting model tends to 
perform better; the optimal linear model produces the best 
results in terms of validity. Although the research has not 
focused on political variables and the criterion variables 
have been short-term in nature, the evidence for integrating 
judgments and modeling is impressive. 

Slater and Longo (1980) refer to the potential applica¬ 
bility of bootstrapping to political risk assessment. Simi¬ 
larly, the synthesis of expert judgments and modeling makes 
sense in the context of "I&W risk assessment." 

Furthermore, other analogies between corporate risk 
assessment and intranational I&W can be drawn. For example, 
Gebelein et al. (1978) view political risk from a component 
perspective. Political risk is a global concept which sub¬ 
sumes civil disorder losses, external war losses, sudden 
expropriation, creeping expropriation, taxation changes, etc. 

The I&W analyst is similarly concerned with a range of 
discrete risks; risk in general is equivalent to objective 
doubt regarding the outcome of a situation and can therefore 
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be regarded (and expressed) probabilistically (see Haendel, 
1979). There are "risks" relating to illegal elite change 
(e.g., coups), unexpected mass-based regime change (e.g., 
revolutions), ongoing internal war, episodic unrest (terror¬ 
ism, fluctuating communa-i instability, etc.), and lower-level 
instability (e.g., nonviolent collective' protest). There 
are also "risks" associated with various types of policy 
change. Further, second-order crises should also be mon¬ 
itored and assessed (e.g., economic upheavals with subsequent 
possible ramifications in the spheres of policy, regime, 
and/or system change). All of this suggests that intra¬ 
national I&W must take a broad view of political change and 
expand the dominant focus from visible, intense conflict to 
salient crisis, conflict, and change incidents and trends. 

In attempting to assess the multitudinous aspects and 
instances of intranational political change, expert assess¬ 
ments should not be elicited in an ad hoc or unstructured 
manner. The judgment process should be anchored in a 
Bayesian framework. The relevance of Bayesian methodology 
to the I&W process, intelligence analysis in general, and 
forecasting is demonstrated in Andriole (1980), Barclay et 
al. (1977), Duncan and Job (1980), Heuer (1980), Job and 
Duncan (1980), Kelly (1972, 1973), and Kelly et al. (1977). 

The Bayesian approach has been employed in DoD and CIA 
to analyze a number of problems which fall under the rubric 
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of "international politics." Among these are the Indo- 
Pakistan War of 1971, the Soviet invasion of Czechoslovakia, 
Sino-Soviet relations, Syria-Israeli relations, Rhodesia- 
Zimbabwe relations, and the likelihood of a Warsaw Pact 
invasion of the NATO sphere, a North Vietnamese attack on 
South Vietnam, and an Arab-Israeli war in the Middle East. The 
work of Duncan and Job (1980) especially implies that Bayesian 
analysis can be integrated with objective event/interaction 
data and transferred to the realm of intranational I&W; one 
of their two case studies, in fact, was the Rhodesia-Zimbabwe 
crisis arena, a classic example of a protracted civil war 
situation. 

The Duncan-Job work is explicitly concerned with short¬ 
term forecasting? their focus is the probability of transi¬ 
tion from state to state in a dyadic conflict arena (e.g., 
Israel-Syria). A stochastic Markovian methodology is employed 
in conjunction with Bayesian statistical inference procedures 
to produce a series of forecasts about the expected movement 
in a dyadic system (e.g., from overt hostility to war). 

The adaptation of this approach or a variant of it to 
intranational I&W could be explored as a task of the second 
phase of research. The heart of Bayesian analysis—whether 
in the form of "basic Bayes" or "hierarchical Bayes"—is the 
revision of probability judgments on the basis of a stream 
of new, incoming information. Andriole (1980) and Heuer 
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(1980) catalogue a number of heuristic and other advantages 
of a Bayesian approach. Basic Bayes is ideal for short-term 
forecasting problems and hierarchical Bayes for complex, 
long-term problems. 

A variety of both types of problems can be delineated in 
the intranational I&W analytical arena. A classic short¬ 
term situation is the unfolding of policy on a day-to-day 
basis in a period of heightened tension or crisis. More 
generic phenomena (e.g., long-term stabilization) could be 
structured and analyzed in terms of a hierarchical model. The 
nature of the I&W task, the characteristics of intranational 
intelligence analysis, and prior "successes" in interna¬ 
tional I&W all support the idea of experimenting with 
Bayesian inference models as part of the research effort. 

The data banks of comparative and international politics 
have undergone a quantitative proliferation in recent years. 
With little or no additional data collection, we can amass a 
number of relevant data bases for the tracking and analysis 
of intranational affairs in Africa. 

For both descriptive profiling and modeling, contextual 
data are indispensible. In contrast to dynamic event data, 
contextual data refer to trends, situations, and conditions. 
Such data can be used to anchor event data in a contextual 
framework. 
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I&W analysts could find such data useful for background 
purposes and for information display and retrieval purposes. 
When the situation "heats up" in a given country, for example, 
the analyst could retrieve descriptive data pertaining to a 
variety of characteristics. What are the recent economic 
trends? What are the major tribal and other ethnocultural 
groupings in the system? What is revealed by qualify of life 
and system performance indices? 

For explanatory purposes, such data would be useful for 
examining the potential determinants of phenomena reflected 
in event data patterns and trends. Using World Handbook 
and similar data sets, it should be possible to construct 
and test a variety of models. Such models could be analyzed 
"in-house" by the contractors; if they were useful, they 
could be integrated into the intranational system and made 
available to users (with esoteric aspects made transparent). 
Examples of contextual data include: 

• Economic data 

• Attribute data 

• Ethnicity data 

Regarding the first, data from the Economic Series of 
the African Research Bulletin and other appropriate data 
( London Financial Times , etc.) could be amassed; the nexus 
between economic and political trends and states of affairs 
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could be illuminated. Rapkin and Avery (1980), Smythe 
(1980), and many others demonstrate the impact of economic 
phenomena on intrasocietal politics and stress. 

There are various sociopolitical, demographic, and 
cultural attribute data sets, including one for 77 countries 
and 10 years collected under the auspices of DARPA/CTD (see 
Hopple, 1978; Wilkenfeld et al., 1980). Especially useful 
should be the recently updated World Handbook data and the 
Black Africa Handbook data. 

The central role of ethnicity and nationality in the 
study of inter- and intranational politics is obvious (see 
e.g., Nielsson, 1980). Further, Africa is the most hetero¬ 
geneous region in the world in terms of societal pluralism; 
many African nations have extensive tribal and other 
cleavages. For studying political change and stability, 
more use should be made of available ethnicity data. 
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APPENDIX A 


SAMPLE OUTPUT 
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This section provides a detailed overview of 
the key elements of the analytic EWAMS, using 
the case of Ethiopia-Somalia for the time 
period from 770101 to 780630. 
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The data base graph shows which sources are 
preferred for all JCS regions. NYT (New York 
Times ) and MAG (Manchester Guardian ) data are 
available for all 7 regions; FBIS (Foreign 
Broadcast Information Service) data exist 
only for Africa. 




I 
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These are the "input parameters" for the 
analysis: actors; activity flow; time 

increment; time parameters (start and end 
dates). All actors have three-letter codes 
(eth is Ethiopia and som is Somalia); see 
the Appendix for a complete list. 
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Special purpose indicators is a special 
feature of crisis; it is discussed in 
Hopple and Pulley (1979). See the Research 
Memorandum section of the Publications list 
in Appendix B. 
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PROCESSING 

COMPLETED 


Nunbcr of «v«nt* found: 163 

Do mou d*«ir« conflict probabi 1 i ti«*T<y''b) 1 y 



A total of 163 events is found 
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Tabular data are requested. 
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Event frequencies for the 10 cooperation 
categories are shown in this frame. 
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The user now requests plots, which show 
the tabular data in a graphic format. 








This is the basic plots menu. Here the 
user asks for a multiplot (#1), which 
will show total activity, cooperation, and 
conflict on a single graph. 
















The crisis in July and the war in August 
are clearly revealed. 












ETHIOPIA 


SOMALIA 



Tension rose in early 1977 (February), in¬ 
creasing again in June and especially July; 
the war continued through 1977 and into 1978, 
accounting for the uniformly high tension 














Now the user requests four shaded bar-chart; 
(conflict, conflict Z-score, tension, tensi< 
Z-score), 


























The change option is used to modify the 
input parameters. 
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By using the time parameter change option, 
recent Ethiopia-Somalia interactions can be 
tracked. August and September 1980 weekly 
data are specified. 
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Number of events found: 
Press RETURN to continue! 















********** TEXT «»»**«**»» 


Picas* Specify Event Types: 

1. fill cooperative events 

2. All conflictual events 

i. ttinnsu _j_ 

Set tine parenetersC730101-?71231>: 800001-800930 



The user asks for #2 (all conflictual events) 
Note that start and end dates must always be 
specified in the text option. 
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This and the following frame display the 
narrative text. 
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Events: All Conflict Sourc 

Actors: ETHIOPIA<<->>SOMALIA 

Dates: 809801 - 800930 

date: 800828 actor: S30 event: 121 tar«et: 320 arena: 0 

ettv'son Ethiopia accuses Sonalia of fabricating reports of 
an Ethiopian attack in order to receive wore USA aid. 

date: 800901 actor: 530 event: 121 target: 520 arena: 0 

cth'son ETH assails the treaty between the USA and Sonalia 
as a threat to the stability to the Horn of Africa. 


date: 800921 actor: 530 event: 223 target: 520 arena: 
cth/'son Ethiopian Nig-23's bonb and strafe targets 
in Sonalia. 

data: 800921 actor: 320 event: 121 target: 330 arena: 
soM/’eth Sonalia accuses Ethiopia of using planes to 
bonb and strafe targets in Sonalia. 

Do uou want to look at wore text <y'n>:n 














Menu returns the user to the central 
EWAMS menu. 
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The user decides to terminate execution 
and therefore types in end . 
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EARLY WARNING AND MONITORING PROJECT 


LIST OF PUBLICATIONS* 


INTERNATIONAL EARLY WARNING AND MONITORING: 
RECENT RESEARCH MEMORANDA 


RM 79-1-36 


RM 79-1-37 


RM 79-1-38 


RM 79-1-39 


RM 79-1-40 


RM 79-1-41 


The Region Option in the Early Warning and 
and Monitoring System: Incorporating 
National Characteristics Regions 
Gerald W. Hopple 
January, 1979 

Real-Time Performance Evaluation Tests: Initial 
Findings and Further Tests 
Paul J. Rossa 
January, 1979 

Cross-National Crisis Indicators (CNCI) Data: 

State Sample, Variables, and Indices 
Gerald W. Hopple, Paul J. Rossa 
January, 1979 

Background Intelligence Data and the 

Government Foreign Affairs Analyst: An 
Overview 

Gerald W. Hopple, Paul J. Rossa 
January, 1979 

Crisis Forecasting Under Uncertainty: 

Proposal for a Two-Stage Forecasting 
Approach 
Thomas R. Davies 
February, 1979 


The Times of London/Manchester Guardian Data 
Basel Coding Procedures, Coder Reliability, 
and Quality Control 
Paul J. Rossa, Gerald W. Hopple 
February, 1979 


This publications list includes papers produced by past 
and current analysts on the Early Warning and Monitoring 
Project. 
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RM 79-1-42 


The Times of London and the Ne w York Times as 
Sources for the World Event Interaction Survey 
(WEIS) Data Base: A Comparative Profile 
Gerald W. Hopple 
February, 1979 

RM 79-1-43 The Real-Time Performance of the Early Warning 
and Monitoring System: A Preliminary Assess¬ 
ment 

Paul J. Rossa 
February, 1979 

RM 79-1-44 Recent Developments and Future Directions in 
International Crisis Research [Discussion 
Draft] 

Gerald W. Hopple, Paul J. Rossa 
March, 1979 

RM 79-1-45 The Intelligent I&W Module in the Early Warning 
and Monitoring System: An Integrated Design 
Gerald W. Hopple, Paul J. Rossa, Brenda Bell 
March, 1979 

RM 79-1-46 Indications, Probabilities, and Thresholds: 

Researching and Integrating the Political 
Component of the Early Warning and Monitoring 
System 

Paul J. Rossa 
March, 1979 

RM 79-1-47 The Manchester Guardian Data Base as an Option 
in the Early Warning and Monitoring System: 

A Prolegomenon 
Gerald W. Hopple 
March, 1979 

RM 79-1-48 The Diffusion of Political Conflict: Experi¬ 
mental Tests of a New Crisis Indicator 
Thomas R. Davies 
March, 1979 

RM 79-1-49 Patterns Underlying Superpower Involvement in 
African Conflicts, 1966-1978 
Paul J. Rossa, Gerald W. Hopple 
April, 1979 

RM 79-1-50 Economic Indicators in the Early Warning and 
Monitoring System: A Status Report 
Gerald W. Hopple, Michael A. Daniels 
April, 1979 

RM"79-1-51 Intelligent I&W for the Early Warning and 
Monitoring System 
Paul J. Rossa 
May, 1979 
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RM 79-1-52 

RM 79-1-53 

RM 79-1-54 

RM 79-1-55 

RM 79-1-56 

RM 79-1-57 

RM 79-1-58 

RM 79-1-59 

RM 79-1-60 

RM 79-1-61 

RM 79-1-62 


ROZ: An Update Report 
Gerald W. Hopple 
May, 1979 

Probabilistic Forecasts of International 
Crisis Through Discriminant Functions 
Paul J. Rossa 
June, 1979 

Historical Data for Crisis Analysis: World 
War II, The Korean War, and the Cuban Missile 
Crisis 

Gerald W. Hopple 
June, 1979 

Crisis Warning Through Unusualness Indicators: 

Comparing Alternative Historical Tails 
Paul J. Rossa 
June, 1979 

The Manchester Guardian Data Base: Intelligent 
IS.W Tests 
Gerald W. Hopple 
June, 1979 

Special Purpose Indicators: Examples and 
Utility 

Gerald W. Hopple and Marshall Pulley 
July, 1979 

Understanding Probabilistic Crisis Forecasts 
Paul J. Rossa 
July, 1979 

The Country Activity Profile Indicator: A 
Comparison of 1976-1977 and 1978-1979 
Gerald W. Hopple and Norman G. Snyder 
August, 1979 

International Crisis Analysis: Recent 
Developments and Future Directions 
Gerald W. Hopple 
August, 1979 

The Keyword Option in the Analytic Early 
Warning and Monitoring System 
Marshall Pulley and Gerald W. Hopple 
September, 1979 

The Early Warning and Monitoring System Software 
A Status Report 

Marshall L. Pulley and Norman G. Snyder 
October, 1979 
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RM 79-1-63 


Evaluating Potential EWAMS Systemic Scans: 
The Event Flow Indicator (EFI) and the 
Pattern Recognition Algorithm (.PRA) 
Frederick A. Rothe 
December, 1979 


RM 80-1-64 


RM 80-1-65 


RM 80-1-66 


RM 80-1-67 


RM 80-1-68 


RM 80-1-69 


RM 80-1-70 


RM 80-1-71 


Two Windows on the World: The Manchester Guardian 
and New York Times as Sources for the Early 
Warning and Monitoring System 
Gerald W. Hopple 
January, 1980 

Threat Networks: Digraph Theory and the 1971 
India-Pakistan War 
Frederick A. Rothe 
January, 1980 

Early Warning and Monitoring System Performance: 

The Iranian Embassy Crisis of 1979 
Paul J. Rossa 
January, 1980 

Crisis Probabilities in the Early Warning and 
Monitoring System: Use in Indications and 
Warnings 
Paul J. Rossa 
January, 1980 

Daily and Weekly Indicators in the Early Warning 
and Monitoring System: Use in Indications and 
Warnings 
Paul J. Rossa 
January, 1980 

Evaluating the Performance of the Early Warning 
and Monitoring System: A Design for Real- 
Time Testing and Assessment 
Frederick A. Rothe 
February, 1980 

Weekly Indicators for Crisis Warning: A Retro¬ 
spective Study of Probabilistic Forecasting 
Potential in the Early Warning and Monitoring 
System 

Paul J. Rossa 
February, 1980 

Further Exploration of Threat Networks: The 1979 
Iran and Afghanistan Episodes 
Frederick A. Rothe 
April, 1980 
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RM 80-1-72 


Crisis Warning: A Comparison of the Manchester 
Guardian and New York Times 
Gerald W. Hopple 
April, 1980 

RM 80-1-73 Moving Average Daily Indicators for Discriminant 
Function Probabilistic Crisis Forecasts 
Paul J. Rossa 
May, 1980 

RM 80-1-74 Automated Crisis Warning and Monitoring: 

Exploring a Staircase Display Option 
Gerald W. Hopple 
September, 1980 


INTRANATIONAL EARLY WARNING AND MONITORING: 
RECENT RESEARCH MEMORANDA 


RM 79-3-1 An Intranational I&W Warning and Monitoring 
System for Africa: An Overview 
Gerald W. Hopple 
November, 1979 

RM 79-3-2 An Intrastate Political Events Survey: A Data 
Collection Design 
Paul J. Rossa 
December, 1979 

RM 80-3-3 The GEN2 System for Inter- and Intra-State Events 
Data 

Paul J. Rossa and Norman G. Snyder 
January, 1980 

RM 80-3-4 Indicator Development for Intrastate Early 
Warning and Monitoring 
Paul J. Rossa 
March, 1980 

RM 80-3-5 Strategies for the Analysis of Intranational 
Event Data 

Gerald W. Hopple and Robert 0. Slater 
June, 1980 


TECHNICAL REPORTS 


TR 76-19 Progress Report on the Development of an 
Integrated Crisis Warning System 
Stephen J. Andriole 
December, 1976 
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TR 80- 


Books 
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Software Design for an Interactive Crisis Early 
Warning Prototype System 
James F. Wittmeyer 
December, 1976 

*21 Crisis and Crisis Forecasting: An Analytical 
Review of the Literature 
Richard W. Parker 
December, 1976 

•17-39 The Early Warning and Monitoring System: A 
Progress Report 

Judith Ayres Daly and Thomas R. Davies 
July, 1978 

-3-39 User's Guide to the Early Warning and Monitoring 
System 

Thomas R. Davies and Brenda D. Bell 
November, 1978 

-A-l User's Guide to the Early Warning and Monitoring 
System 

Gerald W. Hopple and Brenda D. Bell 
July, 1979 

-A-2 European Command/Early Warning and Monitoring 
System User's Guide 
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March, 1980 
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Gerald W. Hopple and Paul J. Rossa 
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•A-4 Internal and External Crisis Early Warning and 

Monitoring 
Gerald W. Hopple 
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Expert-Generated Data: Applications in 
International Affairs 

Gerald W. Hopple and James A. Kuhlman (co-editors) 
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Conceptualizing an Integrated Crisis Warning 
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Stephen J. Andriole and Robert A. Young 
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Journal of Defense Research 
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Computerized Crisis Forecasting 
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Recognizing Patterns of International Political 
Situations 
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Research on the international Early Warning and Monitoring System 
(EWAMS) and the intranational Africa Warning and Monitoring System (AWAMS) 
is the subject of this report. Recent political indicator analysis and 
testing and software development and refinement activities have been exten¬ 
sive in scope and magnitude. The master version of the EWAMS has been fully 
developed and is being tested and enhanced. 

A key objective of recent research has been the realm of crisis warning. 
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The discriminant analysis methodology was tested extensively and 
has been incorporated into the system. The accuracy and efficiency of 
the methodology led to a decision to replace the simple extrapolative 
warning methodology with one based on discriminant function-generated 
probabilities. 

The prototype Africa system reflects extensive basic research 
on monitoring intranational affairs. The current version of the AWAMS 
features an empirical data base (from several sources), political 
indicators, and a computer base. Thus far, the conceptual blueprint 
has been completed; the data collection design has been developed for 
intranational events, subnational actors and targets, and other ele¬ 
ments of the AWAMS. In addition, the software base of the prototype 
AWAMS has been designed and programmed. 
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